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Abstract 



Numerous formalisms and dedicated algorithms have been designed in the last decades 
to model and solve decision making problems. Some formalisms, such as constraint net- 
works, can express "simple" decision problems, while others are designed to take into ac- 
count uncertainties, unfeasible decisions, and utilities. Even in a single formalism, several 
variants are often proposed to model different types of uncertainty (probability, possibil- 
ity...) or utility (additive or not). In this article, we introduce an algebraic graphical model 
that encompasses a large number of such formalisms: (1) we first adapt previous structures 
from Friedman, Chu and Halpern for representing uncertainty, utility, and expected utility 
in order to deal with generic forms of sequential decision making; (2) on these structures, 
we then introduce composite graphical models that express information via variables linked 
by "local" functions, thanks to conditional independence; (3) on these graphical models, 
wc finally define a simple class of queries which can represent various scenarios in terms 
of observabilities and controllabilities. A natural decision-tree semantics for such queries 
is completed by an equivalent operational semantics, which induces generic algorithms. 
The proposed framework, called the Plausibility- Feasibility- Utility (PFU) framework, not 
only provides a better understanding of the links between existing formalisms, but it also 
covers yet unpublished frameworks (such as possibilistic influence diagrams) and unifies 
formalisms such as quantified boolean formulas and influence diagrams. Our backtrack 
and variable elimination generic algorithms are a first step towards unified algorithms. 

1. Introduction 

In the last decades, numerous formalisms have been developed to express and solve decision 
making problems. In such problems, an agent must make decisions consisting of either 
choosing actions and ways to fulfill them (as in action planning, task scheduling, or resource 
allocation), or choosing explanations of observed phenomena (as in diagnosis or situation 
assessment). These choices may depend on various parameters: 

1. uncertainty measures, which we call plausibilities, may describe beliefs about the state 
of the environment; 

2. preconditions may have to be satisfied for a decision to be feasible; 
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3. the possible states of the environment and decisions do not generally have the same 
value from the decision makers point of view. Utilities can be expressed to model costs, 
gains, risks, satisfaction degrees, hard requirements, and more generally, preferences; 

4. when time is involved, decision processes may be sequential and the environment may 
be partially observable. This means that there may be several decision steps, and that 
the values of some variables may be observed between two steps, as in chess where 
each player plays in turn and can observe the move of the opponent before playing 
again; 

5. there may be adversarial or collaborative decision makers, each of them controlling a 
set of decisions. Hence, a multi-agent aspect can yield partial controllabilities. 

Given the plausibilities defined over the states of the environment, the feasibility con- 
straints on the decisions, the utilities defined over the decisions and the states of the environ- 
ment, and given the possible multiple decision steps, the objective is to provide the decision 
maker with optimal decision rules for the decision variables he controls, depending on the 
environment and on other agents. To be concise, the class of such problems is denoted as 
the class of sequential decision problems with plausibilities, feasibilities, and utilities. 

Various formalisms have been designed to cope with problems of this class, sometimes 
in a degenerated form (covering only a subset of the features of the general problem) : 

• formalisms developed in the boolean satisfiability framework: the satisfiability prob- 
lem (SAT), quantified boolean formulas, stochastic SAT (Liftman, Majercik, &; Pitassi, 
2001), and extended stochastic SAT (Liftman et al., 2001); 

• formalisms developed in the very close constraint satisfaction framework: constraint 
satisfaction problems (CSPs, Mackworth, 1977), valued/semiring CSPs (Bistarelli, 
Montanari, Rossi, Schiex, Verfaillie, &: Fargier, 1999) (covering classical, fuzzy, addi- 
tive, lexicographic, probabilistic CSPs), mixed CSPs and probabilistic mixed CSPs 
(Fargier, Lang, & Schiex, 1996), quantified CSPs (Bordeaux & Monfroy, 2002), and 
stochastic CSPs (Walsh, 2002); 

• formalisms developed to represent uncertainty and extended to represent decision 
problems under uncertainty: Bayesian networks (Pearl, 1988), Markov random fields 
(Chellappa & Jain, 1993) (also known as Gibbs networks), chain graphs (Fryden- 
berg, 1990), hybrid or mixed networks (Dechter & Larkin, 2001; Dechter & Mateescu, 
2004), influence diagrams (Howard & Matheson, 1984), unconstrained (Jensen & Vom- 
lelova, 2002), asymmetric (Smith, Holtzman, & Matheson, 1993; Nielsen & Jensen, 
2003), or sequential (Jensen, Nielsen, & Shenoy, 2004) influence diagrams, valuation 
networks (Shenoy, 1992), and asymmetric (Shenoy, 2000) or sequential (Demirer & 
Shenoy, 2001) valuation networks; 

• formalisms developed in the classical planning framework, such as STRIPS plan- 
ning (Fikes & Nilsson, 1971; Ghallab, Nau, & Traverso, 2004), conformant plan- 
ning (Goldman & Boddy, 1996), and probabilistic planning (Kushmerick, Hanks, &; 
Weld, 1995); 
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• formalisms such as Markov decision processes (MDPs), probabilistic, possibilistic, 
or using Spohn's epistemic beliefs (Spolm, 1990; Wilson, 1995; Giang & Shenoy, 
2000), factored or not, possibly partially observable (Puterman, 1994; Monahan, 1982; 
Sabbadin, 1999; Boutilier, Dean, & Hanks, 1999; Boutilier, Dearden, & Goldszmidt, 
2000). 

Many of these formalisms present interesting similarities: 

• they include variables modeling the state of the environment (environment variables) 
or the decisions (decision variables); 

• they use sets of functions which, depending on the formalism considered, can model 
plausibilities, feasibilities, or utilities; 

• they use operators either to combine local information (such as x to aggregate proba- 
bilities under independence hypothesis, + to aggregate gains and costs), or to project 
a global information (such as + to compute a marginal probability, min or max to 
compute an optimal decision). 

Even if the meaning of variables, functions, and combination or projection operators 
may be specific to each formalism, they can all be seen as graphical models in the sense that 
they exploit, implicitly or explicitly, a hypergraph of local functions between variables. This 
article shows that it is possible to build a generic algebraic framework subsuming many of 
these formalisms by reducing decision making problems to a sequence of so-called "variable 
eliminations" on an aggregation of local functions. 

Such a generic framework will be able to provide: 

• a better understanding of existing formalisms: a generic framework has an obvious 
theoretical and pedagogical interest, since it can bring to light similarities and dif- 
ferences between the formalisms covered and help people of different communities to 
communicate on a common basis; 

• increased expressive power: a generic framework may be able to capture problems 
that cannot be directly modeled in any existing formalism. This increased expressive- 
ness should be reachable by capturing the essential algebraic properties of existing 
frameworks; 

• generic algorithms: ultimately, besides a generic framework, it should be possible 
to define generic algorithms capable of solving problems defined in this framework. 
This objective fits into a growing effort to identify common algorithmic approaches 
that were developed for solving different AI problems. It may also facilitate cross- 
fertilization by allowing each subsumed framework to reuse algorithmic ideas defined 
in another one. 

1.0.1 Article Outline 

After the introduction of some notations and notions, the article starts by showing, with a 
catalog of existing formalisms for decision making, that a generic algebraic framework can 



423 



PrALET, VERFAILLIE, & SCHIEX 



be informally identified. This generic framework, called the Plausibility-Feasibility-Utility 
(PFU) framework, is then formally introduced in three steps: (1) algebraic structures cap- 
turing plausibilities, feasibilities, and utilities are introduced (Section 4), (2) these alge- 
braic structures are exploited to build a generic form of graphical model (Section 5), and 
(3) problems over such graphical models are captured by the notion of queries (Section 6). 
The framework is analyzed in Section 7 and generic algorithms are defined in Section 8. 
A table recapitulating the main notations used is available in Appendix A and the proofs 
of all propositions and theorems appear in Appendix B. A short version of the framework 
described in this article has already been published (Pralet, Verfaillie, & Schiex, 2006c). 

2. Background Notations and Definitions 

The essential objects used in the article are variables, domains, and local functions (called 
scoped functions). 

Definition 1. The domain of values of a variable x is denoted dom{x) and for every 
a € dom{x), {x,a) denotes the assignment of value a to x. By extension, for a set of 
variables S, we denote by dom(S) the Cartesian product of the domains of the variables in 
S, i.e. dom{S) = JlxeS ^^"^(^)' element A of dom{S) is called an assignment of S.^ 

IfAi, A2 are assignments of disjoint subsets Si, S2, then A1.A2, called the concatenation 
of Ai and A2, is the assignment of Si U S2 where variables in Si are assigned as in Ai and 
variables in S2 are assigned as in A2. If A is an assignment of a set of variables S, the 
projection of A onto S' C S is the assignment of S' where variables are assigned to their 
value in A. 

Definition 2. (Scoped function) A scoped function is a pair {S,f) where S is a set of 
variables and ip is a function mapping elements in dom(S) to a given set E. In the following, 
we will often consider that S is implicit and denote a scoped function (S, (p) as ip alone. The 
set of variables S is called the scope of ip and is denoted sc{ip). If A is an assignment of a 
superset of sc{ip) and A' is the projection of A onto sc{ip), we define (p{A) by ip{A) = ip{A'). 

For example, a scoped function ip mapping assignments of sc{ip) to elements of the 
boolean lattice B = {t, /} is analogous to a constraint describing the subset of dom{sc{(p)) 
of authorized tuples in constraint networks. 

From this, the general notion of graphical model can be defined: 

Definition 3. ( Graphical model) A graphical model is a pair {V, $) where V = {xi, . . . , x„} 
is a finite set of variables and ^ = {pi, . . . , ipm} is a finite set of scoped functions whose 
scopes are included in V. 

The terminology of graphical models is used here simply because a set of scoped functions 
can be represented as a hypergraph that contains one hyperedge per function scope. As 
we will see, this hypergraph captures a form of independence (see Section 5) and induces 
parameters for the time and space complexity of our algorithms (see Section 8). This 
definition of graphical models generalizes the usual one used in statistics, defining a graphical 

1. An assignment of S = {xi, . . . ,Xk} is actually a set of variable-value pairs {{xi,ai), . . . , {xk,ak)}; we 
assume that variables are implicit when using a tuple of values (ai, . . . , Ofc) G dom{S). 
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model as a (directed or not) graph where the nodes represent random variables and where 
the structure captures probabilistic independence relations. 

"Local" scoped functions in a graphical model give a space-tractable definition of a 
global function defined by their aggregation. For example, in a Bayesian network (Pearl, 
1988) a global probability distribution Px.y,z over x, y, z may be defined as the product 
(using operator x) of a set of scoped functions {P^, Py\x, Pz\y}- Local scoped functions 
can also facilitate the projection of the information expressed by a graphical model onto a 
smaller scope. For example, in order to compute a marginal probability distribution Vy^z 
from the previous network, we can compute Ylx ^x,y,z = ^ ^y\x) ^ Pz\y and avoid 

taking P^^y into account. Here the operator ^ is used to project information onto a smaller 
scope by eliminating variable x. Operators used to combine scoped functions will be called 
combination operators, while operators used to project information onto smaller scopes will 
be called elimination operators. 

Definition 4. (Combination) Let ipi, ip2 be scoped functions to Ei and E2 respectively. Let 
<^ : EiX E2 ^ E be a binary operator. The combination ofipi and (p2, denoted by ipi<^ip2, is 
the scoped function to E with scope sc{(pi)\J sc{ip2) defined by {ipi®ip2){A) = ipi[A) ® ip2{A) 
for all assignments A of sc{ipi) U sc{ip2). ® is called the combination operator 0/991 and 

In the rest of the article, all combination operators will be denoted (8). 

Definition 5. (Elimination) Let if be a scoped function to E. Let op be an on E which 
is associative, commutative, and has an identity element . The elimination of variable 
X from ip with op is a scoped function whose scope is sc{ip) — {x} and whose value for an 
assignment A of its scope is {op^ ^){A) = op^g^^^^^.) Lp{A.{x, a)). In this context, op is called 
the elimination operator for x. The elimination of a set of variables S = {xi, . . . , x^} from 
ip is a function with scope sc{ip) — S defined by opg ^p{A) = opj^i^^^^j^f^g^ ip{A.A'). 

Hence, when computing {Px x Py^^ x Pz\x), scoped functions are aggregated using 
the combination operator = x and the information is projected by eliminating x using 
the elimination operator +. In this article, denotes elimination operators. 

In some cases, the elimination of a set of variables S with an operator op from a scoped 
function ip should only be performed on a subset of dom{S) containing assignments that 
satisfy some property denoted by a boolean scoped function F. Then, we must compute 
for every A € dom{sc{ip) — S) the value opy^i^dg^(^g-^ p(^y^i)^-tip{A.A'). For simplicity and 
homogeneity, and in order to always use elimination over dom{S), we can equivalently 
truncate ip so that elements of dom{S) which do not satisfy F are mapped to a special 
value (denoted 0) which is itself defined as a new identity for op. 

Definition 6. (Truncation operator) The unfeasible value () is a new special element that 
is supposed to be outside of the domain E of every elimination operator op : E x E ^ E. 
We explicitly extend every elimination operator op : E x E ^ E on E D {()} by taking the 
convention op{(), e) = op{e, 0) = e for all e £ E U {(}}■ 

Let {t, /} be the boolean lattice. For any boolean b and any e £ E, we define b-k e to be 
equal to e ifb = t and otherwise, -k is called the truncation operator. 
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Given a boolean scoped function F, and make it possible to write quantities like 
oPA'Gdom{S),F{A')=t f as the elimination ops (F-kip). 

In order to solve decision problems, one usually wants to compute functions mapping the 
available information to a decision. The notion of decision rules will be used to formalize 
this: 

Definition 7. (Decision rule, policy) A decision rule for a variable x given a set of variables 
S' is a function 6 : dom{S') dom{x) mapping each assignment of S' to a value in dom{x). 
By extension, a decision rule for a set of variables S given a set of variables S' is a function 
6 : dom{S') — dom{S). A set of decision rules is called a policy. 

Definition 8. ( Optimal decision rule) Consider a totally ^-ordered set E, a scoped function 
if from dom{sc{(p)) to E, and a set of variables S C sc{(p). A decision rule 5 : dom(sc{ip) — 
S) — > dom{S). is said to be optimal iff, for all {A^A') € dom{sc{ip) — S) x dom{S), 
(p{A.5{A)) ^ ip{A.A') (resp. (p{A.5{A)) < (p{A.A')). Such a decision rule always exists 
when dom{sc{ip)) is finite. 

In other words, optimal decision rules are examples of decision rules given by argmin 
and argmax (in this article, we consider that optimality on decision rules is always given by 
min or max on some totally ordered set). 

Definition 9. (Directed Acyclic Graph (DAG)) A directed graph G is a DAG if it contains 
no directed cycle. When variables are used as vertices, padx) denotes the set of parents of 
variable x in G. 

Last, [1,?!] will denote the set of integers i such that 1 < i < n. 

3. Prom Examples of Graphical Models to a Generic Framework 

We now present different AI formalisms for expressing and solving decision problems. In 
the most simple case, a single decision which maximizes utility is sought. The introduction 
of plausibilities (uncertainties), unfeasible actions (feasibilities), and sequential decision 
(several decision steps with some observations between decision steps) only appears in the 
most sophisticated frameworks. The goal of this section is to show that these formalisms 
can all be viewed as graphical models where specific elimination and combination operators 
are used. 

3.1 Examples of Graphical Models 

The examples used cover various Al formalisms, which are briefly described. A wider and 
more accurate review of existing graphical models could be provided (Pralet, 2006). 

3.1.1 Constraint Networks 

Constraint networks (CNs, Mackworth, 1977), often called constraint satisfaction problems 
(CSPs), are graphical models {V, $) where the scoped functions in <I> are constraints mapping 
assignments onto {t,/}. The usual query on a CN is to determine the existence of an 
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assignment of V that satisfies all constraints. By setting f ~< t, this decision problem can 
be answered by computing: 



If this quantity equals true, then an optimal decision rule for V defines a solution. This 
query can be answered by performing eliminations (using max) on a combination of scoped 
functions (using A). Replacing the hard constraints in $ by soft constraints (boolean scoped 
functions being replaced by cost functions) and replacing A by an abstract operator (g) equal 
to +, min, X, . . . leads to queries on a valued and totally ordered semiring CN (Bistarelli 
et al, 1999). 

3.1.2 Bayesian Networks 

Bayesian networks (BNs, Pearl, 1988) can model problems with plausibilities expressed as 
probabilities. A BN is a graphical model {V, <&) in which $ is a set of local conditional 
probability distributions: $ = {Px\paG{x) ■, ^ ^ ^}) where G is a DAG with vertices in V. 
A BN represents a joint probability distribution Py over all variables as a combination of 
local conditional probability distributions {Py = Hxey Px\paG{x))-: J^st as the combination 
of local constraints in a CN defines a global constraint over all variables. One possible query 
on a BN is to compute the marginal probability distribution of a variable y ^V: 



Equation 2 corresponds to variable eliminations (with +) on a product of scoped functions. 
In other queries on BNs such as MAP (Maximum A Posteriori hypothesis), eliminations 
with max are also performed. 

3.1.3 Quantified Boolean Formulas and Quantified CNs 

Quantified boolean formulas (QBFs) and quantified CNs (Bordeaux & Monfroy, 2002) can 
model sequential decision problems. Let xi, X2, 3:3 be boolean variables. A QBF using the 
so-called prenex conjunctive normal form looks like (with f ~<t): 

3xiVx23x3((-iXi V X3) A {x2 V X3)) = maxminmax((-ixi V X3) A (x2 V X3)). (3) 



Thus, the query "Is there a value for xi such that for all values of X2, there exists a value 
for X3 such that the clauses -1X1 V X3 and X2 V X3 are satisfied?" can be answered as in 
Equation 3 using a sequence of eliminations (max over xi, min over X2, and max over X3) 
on a conjunction of clauses. In a quantified CN, clauses are replaced by constraints. 

3.1.4 Stochastic CNs 

A stochastic CN (Walsh, 2002) can model sequential decision problems with probabilities as 
plausibilities and hard requirements as utilities, provided that the decisions do not influence 
the environment (the so-called contingency assumption). In a stochastic CN, two types of 
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variables are defined: decision variables di and environment (stochastic) variables sj. A 
global probability distribution over the environment variables is expressed as a combination 
of local probability distributions. If the environment variables are mutually independent, 
these local probability distributions are simply unary probability distributions Pgj ■ Finally, 
a stochastic CN defines a set of constraints {Ci, . . . , Cm} mapping tuples of values onto 
{0, 1} (instead of {t,f}). This allows constraints to be multiplied with probabilities. 

Consider a situation where first two decisions di and ^2 are made, then an environment 
variable si is observed, then decisions d^ and d/^ are made, while an environment variable ,82 
remains unobserved. A possible query on a stochastic CN is to compute decision rules for 
di,d2,d3, and d^ which maximize the expected constraint satisfaction, through Equation 4: 

S^s^ ^ ^^^^ ^ (a.[iH^O- 

Sl S2 

The answer to the query defined by Equation 4 is determined by a sequence of eliminations 
(max over the decision variables, + over the environment ones) on a combination of scoped 
functions (probabilities are combined using x , constraints are combined using x , since they 
are expressed onto {0, 1} instead of {t, /}, and probabilities are combined with constraints 
using x). 

3.1.5 Influence Diagrams 

Influence diagrams (Howard & Matheson, 1984) can model sequential decision problems 
with probabilities as plausibilities together with gains and costs as utilities. They can be 
seen as an extension of BNs including the notions of decision and utility. More precisely, 
an influence diagram is a composite graphical model defined by three sets of variables 
organized in a DAG G: (1) a set S of chance variables; for each s € S", a conditional 
probability distribution Psipads) of ^ given its parents in G is specified; (2) a set D of 
decision variables; for each d £ D, padd) is the set of variables observed before decision 
d is made; (3) a set T = {ui, . . . ,Um} of utility variables, each of which is associated with 
a utility function Ui = UpaG{ui) of scope padui). Utility variables must be leaves in the 
DAG, and the utility functions define a global additive utility Uq = m] 

The usual problem associated with an influence diagram is to compute decision rules 
maximizing the global expected utility. If we modify the example used for stochastic CNs 
by replacing Pg-^ by Ps^ \ d2 > Ps2 by Ps2 \ 6,1,6,3 ■, arid the constraints Ci , . . . , Cm by the additive 
utility functions C/i, . . . , Um-, then an optimal policy can be obtained by computing optimal 
decision rules for di, d2, ds, and d^ in Equation 5: 

' S2 

Again, the answer to the query defined by Equation 5 can be computed by a sequence 
of eliminations (alternating max- and sum-eliminations) on a combination of scoped func- 
tions (plausibilities combined using x, utilities combined using -|-, plausibilities and utilities 
combined using x). 
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3.1.6 Valuation Networks 

Valuation networks (Shenoy, 1992) can model sequential decision problems with plausibili- 
ties, feasibilities, and utilities, where plausibilities are combined using x and where utilities 
are additive. A valuation network is composed of several sets of nodes and valuations: (1) 
a set D of decision nodes, (2) a set S of chance nodes, (3) a set F of indicator valuations, 
which specify unfeasible assignments of decision and chance variables, (4) a set P of prob- 
ability valuations, which are multiplicative factors of a joint probability distribution over 
the chance variables, and (5) a set U of utility valuations, representing additive factors 
of a joint utility function Uq = Ylu ^u^i- Ai'cs between nodes are also used to define 
the order in which decisions are made and chance variables are observed. If this order is 
di -< (i2 ^ si ^ ^3 (^4 S2i it can be shown that optimal decision rules for di, ^2, d^, d^ 
are defined through Equation 6: 

=?=?(U"")*(n/.)'<(i:/.))- 

Local feasibility constraints are combined using A, and combined with other scoped func- 
tions using the truncation operator -k (cf. Definition 6). 

3.1.7 Finite Horizon Markov Decision Processes 

Finite horizon Markov Decision Processes (MDPs, Puterman, 1994; Monahan, 1982; Sab- 
badin, 1999; Boutilier et al., 1999, 2000) model sequential decision problems with plausi- 
bilities and utilities over a horizon of T time-steps. For every time-step t, a variable st 
represents the state of the environment and a variable dt represents the decision made after 
observing sj. In factored MDPs, several state variables may be used at each time-step. 

In a probabilistic finite horizon MDP, plausibilities over the environment are described by 
local probability distributions Pg^.^^;^ \ st,dt °f being in state st+i given st and dt- The utilities 
over states and decisions are local additive rewards Rst4t-> boolean functions F^^ \ can 
express whether a decision dt is feasible in state st- An optimal policy for each initial state 
si can be computed by the following equation (which is a bit unusual for defining optimal 
policies for a MDP, but is equivalent to the usual form): 

max ©„ max . . . ©„ max ( A -F^JsJ * ®p Pst+i\st,dt\ ®pu \ ®u Rst4t\- (7) 
di S2 d2 ST dr J ytg[i,T[ / \t&[^,T] ) 

Plausibilities are combined using ©p = x , utilities are combined using = +> plausibilities 
and utilities are combined using (8)p„ = x , decision variables are eliminated using max, and 
environment variables are eliminated using 0^ — -\- . The truncation operator enables the 
elimination operators to ignore unfeasible decisions. 

In a pessimistic possibilistic finite horizon MDP (Sabbadin, 1999), probability distri- 
butions Pst+i\st,dt replaced by possibility distributions TTst+i\st,dt^ rewards Rst,dt 
replaced by preferences fJ-stA^ operators used are = ®p = (^u = min and 

(S)pu : {p,u) — > max(l -p,u). 
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3.2 Towards a Generic Framework 

The previous section shows that usual queries in various existing formahsms can be reduced 
to a sequence of variable eliminations on a combination of scoped functions. 

This observation has led to the definition of algebraic MDPs (Perny, Spanjaard, & Weng, 
2005) or to the definition of valuation algebras (Shenoy, 1991; Kolhas, 2003), a generic 
algebraic framework in which eliminations can be performed on a combination of scoped 
functions. However, valuation algebras use only one combination operator, whereas several 
combination operators may be needed to manipulate different types of scoped functions (as 
previously shown). Moreover, valuation algebras can deal with only one type of elimination, 
whereas several elimination operators may be required for handling the different types of 
variables. In valuation networks (Shenoy, 2000), plausibilities are necessarily represented as 
probabilities, and eliminations with min cannot be performed. Essentially, a more powerful 
framework is needed. 

In order to be simple and yet general enough to cover the queries defined by Equations 1 
to 7, the generic form we should consider is: 



where (1) A is used to combine local feasibilities, ®p is used to combine plausibilities, 
is used to combine utilities, ®pu is used to combine plausibilities and utilities, and the 
truncation operator -k is used to ignore unfeasible decisions without having to deal with 
elimination operations over restricted domains;^ (2) P, U are (possibly empty) sets 
of local feasibility, plausibility, and utility functions respectively; (3) Sov is an operator- 
variable(s) sequence, indicating how to eliminate variables. Sov involves min or max to 
eliminate decision variables and an operator ®u to eliminate environment variables. 

Equation 8 is still very informal. To define it formally, and to provide it with clear 
semantics, we need to define three key elements: 

1. We must capture the essential properties of the combination operators ®u-, ®pu 
used respectively to combine plausibilities, utilities, and plausibilities with utilities. 
We must also characterize the elimination operators (Bu and ©p used to project in- 
formation coming from utilities and plausibilities. These operators will define the 
algebraic structure of the PFU (Plausibility-Feasibility-Utility) framework. 

2. On this algebraic structure, we must define a generic form of graphical model, involv- 
ing a set of variables and sets of scoped functions expressing plausibilities, feasibilities, 
and utilities (sets P, -F, U). Together, they will define a PFU network. The factored 
form offered by such graphical models must also be analyzed in order to understand 
when and how it can be applied to concisely represent global functions (using the 
notion of conditional independence). 

2. In Equation 8, all plausibilities are combined using the same operator ®p and all utilities are combined 
using the same operator ®u\ we denote such models as composite graphical models because they include 
different types of scoped functions (plausibilities, feasibilities, and utilities). Beyond this, Equation 8 
also allows for heterogeneous information among each type of scoped functions. For example, in order to 
manipulate both probabilities and possibilities, we can use ®p defined over probability-possibility pairs 
by (pi,7ri) (g)p (p2,vr2) = (pi X p2, min(7ri, 7r2)). 
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3. Finally, we must define queries on PFU networks capturing interesting decision prob- 
lems. As Equation 8 shows, such queries will be defined by a sequence Sov of operator- 
variable(s) pairs, applied on the combination of the scoped functions in the network. 
The fact that the answer to such queries represents meaningful values from the deci- 
sion theory point of view will be proved by relating approach. 

3.3 Summary 

We have informally shown that several queries in various formalisms dealing with plau- 
sibilities and/or feasibilities and/or utilities reduce to sequences of variable eliminations 
applied to combinations of scoped functions, using various operators. They can intuitively 
be covered by Equation 8. 

The three key elements (an algebraic structure, a PFU network, and a sequence of vari- 
able eliminations) needed to formally define and give sense to this equation are introduced 
in Sections 4, 5, and 6. 

4. The PFU Algebraic Structures 

The first element of the PFU framework is an algebraic structure specifying how the in- 
formation provided by plausibilities, feasibilities, and utilities is combined and synthesized. 
This algebraic structure is obtained by adapting previous structures defined by Friedman, 
Chu, and Halpern (Friedman &: Halpern, 1995; Halpern, 2001; Chu & Halpern, 2003a) for 
representing uncertainties and expected utilities. 

4.1 Definitions 

Definition 10. {E, ®) is a commutative monoid iff E is a set and ® is a binary operator 
on E which is associative (x ® {y ® z) = {x ® y) ® z), commutative (x®y = y®x), and 
has an identity 1e € E (x ® 1e = Ie ® x = x). 

Definition 11. {E,®,0) is a commutative semiring iff 

• (E,®) is a commutative monoid, with an identity denoted Oe, 

• (E,®) is a commutative monoid, with an identity denoted 1e, 

• Oe is annihilator for ® (x ® Oe = Oe), 

• ® distributes over ® (x ® {y ® z) = {x ® y) ® (x ® z) ). 

Definition 12. Let {Ea,®a,®a) be a commutative semiring. Then, (-Efo, 0?,, CSafe) is a semi- 
module on {Ea,®a,(^a) iff 

• {Eh,®h) is a commutative monoid, with an identity denoted Oe^, 

• ®ab ■ Ea X Eh ^ Eb satisfies 

- ®ab distributes over ®h (a ®ab (h ®b h) = {a ®ab bi) ®b {a ®ab h)), 

- ®ab distributes over ®a ({ai ®a 0,2) ®ab b = (ai ®ab b) ®b (a2 ®ab b)), 
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- linearity property: ai 'S>ab (02 ®ab b) = (ai ®a ^2) 'S>ab b, 

- for all be Eh, 0^, ®ab b = and Ie^ '^ab b = b. 

Definition 13. Let E be a set with a partial order :<. An operator ® on E is monotonic 
iff {x ^ y) ^ {x ® z < y ® z) for all x,y, z £ E. 

4.2 Plausibility Structure 

Various forms of plausibilities exist (Halpern, 2003). The most usual one is probabilities. As 
shown previously, for example with Equation 2, probabilities are aggregated using 0p = x 
as a combination operator, and projected using 0p = + as an elimination operator. 

But plausibilities can also be expressed as possibility degrees in [0,1]. Possibilities are 
eliminated using ©p = max and usually combined using (S)p = min. An interesting case 
appears when possibility degrees are booleans describing which states of the environment 
are completely possible or impossible. Plausibilities are then combined using ©p = A and 
eliminated using ©p = V. 

Another example is Spohn's epistemic beliefs, also known as /t-rankings (kappa rank- 
ings, Spohn, 1990; Wilson, 1995; Giang & Shenoy, 2000). In this case, plausibilities are 
elements of NU{-|-cxd} called surprise degrees, is associated with non-surprising situations, 
+00 is associated with completely surprising (impossible) situations, and more generally a 
surprise degree k can be viewed as a probability of e'^' for an infinitesimal e. Surprise degrees 
are combined using ©p = -|- and eliminated using ©p = min. 

To capture these various plausibility modeling frameworks, we start from Friedman- 
Halpern's work on plausibility measures (Friedman & Halpern, 1995; Halpern, 2001). Wey- 
dert (1994) and Darwiche-Ginsberg (1992) developed similar approaches. 

Priedman-Halpern's structure Assume we want to express plausibilities over the as- 
signments of a set of variables S. Each subset of dom{S) is called an event. Friedman and 
Halpern (1995) define plausibilities as elements of a set Ep called the plausibility domain. 
Ep is equipped with a partial order ^p and with two special elements Op and Ip satisfying 
Op <p p <p Ip for QSS.peEp. A function PI : 2'^°'^(^) ^ Ep is a plausibility measure over S 
iff it satisfies P/(0) = Op, Pl{dom{S)) = Ip, and (T^i C W2) (PliWi) <p Pl{W2)). This 
means that Op is associated with impossibility, Ip is associated with the highest plausibility 
degree, and the plausibility degree of a set is as least as high as the plausibility degree of 
each of its subsets. 

Among all plausibility measures, we focus on so-called algebraic conditional plausibility 
measures, which use abstract functions ©p and ®p which are analogous to + and x for 
probabilities. These measures satisfy properties such as decomposability: for all disjoint 
events Wi, W2, Pl{Wi U W2) = PliWi) ©p Pl{W2). As U is associative and commutative, 
it follows that ©p is associative and commutative on representations of disjoint events, i.e. 
(a (Bp b) (Bp c = a (Bp {b (Bp c) and a ffip 6 = 6 ©p a if there exist pairwise disjoint sets 
Wi,W2,W3 such that Pl{Wi) = a, Pl{W2) = b, Pl{W3) = c. More details are available in 
Friedman-Halpern's references (Friedman & Halpern, 1995; Halpern, 2001). 

Restriction of Friedman-Halpern's structure An important aspect in Friedman- 
Halpern's work is that the algebraic properties of (Bp and ®p hold only on the domains 
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of definition of 0p and 0^. Although this is sufficient to express and manipulate plausibili- 
ties, it can be algorithmically restrictive. Indeed, consider a Bayesian network involving two 
boolean variables {xi,X2} and define Pxi,x2 1 XI ■ Assume that P^i is a constant 

factor ipQ = 0.5. In order to evaluate Px2{{x2,t)), the quantity ipQ x P^^ | ^.^ ((x2, i)) must 
be computed. To do so, it is simpler to factor it and compute 930 x ^xi Px2\xx{{x2-,t)). If 
Px2 \ xi{{x2,t).{xi,t)) = 0.6 and Px^\r^^{{x2,t).{xi, f)) = 0.8, the answer is 0.5 x (0.6 + 0.8) = 
0.7. Performing 0.6 + 0.8 requires applying addition outside of the range of usual probabil- 
ities, for which a ©p 6 is defined only if a + 6 < 1, since two probabilities whose sum exceeds 
1 cannot be associated with disjoint events. 

To take such issues into account, we adapt Friedman-Halpern's Ep, ©p, ®p so that ©p 
and ®p become closed in Ep and so that Friedman-Halpern's axioms hold in the closed 
structure. Once this closure is performed, we obtain a plausibility structure. 

Definition 14. A plausibility structure is a tuple {Ep,(Bp,'S>p) such that 

• {Ep,(Bp,0p) is a commutative semiring (identities for (Bp and (^p are denoted Op and 
Ip respectively), 

• Ep is equipped with a partial order :<p such that Op = min(£'p) and such that (Bp and 
^p are monotonic with respect to :<p. 

Elements of Ep are called plausibility degrees 

Note that Ip is not necessarily the maximal element of Ep. For probabilities, Friedman- 
Halpern's structure would be ([0,1],+', x), where a +' b = a + b ii a + b < 1 and is 
undefined otherwise. In order to get closed operators, we take {Ep,(Bp,(Bp) = (M^,+, x) 
and therefore Ip = 1 is not the maximal element in Ep. In some cases, Friedman-Halpern's 
structure is already closed. This is the case with Ac-rankings (where {Ep,(Bp,'^p) = (N U 
{+00}, min, +)) and with possibilities (where (£^p, ©p, (8)p) is typically ([0, 1], max, min), 
although other choices like ([0, l],max, x) are possible). 

Given two plausibility structures {Ep, (Bp, (8ip) and {Ep, ©p, (g)p), if we define E = EpX Ep, 

{PuP'i) ffi {P2,P2) = {Pi ®pP2,p'i ®'pP'2) and {pi,p'i) ®{p2,p'2) = {Pi ®pP2,p'i ®pP2)> then 
{E, ffi, ©) is a plausibility structure too. This allows us to deal with different kinds of plausi- 
bilities (such as probabilities and possibilities) or with families of probability distributions. 

4.2.1 From Plausibility Measures to Plausibility Distributions 

Let us consider a plausibility measure (Friedman & Halpern, 1995; Halpern, 2001) PI : 
2dom{s) _^ o^g^ ^ Qf variables 5. Assume that Pl{Wi U W2) = Pl{Wi) (Bp PKW2) 
for ah disjoint sets Wi,W2 G 2^°™(^), as is the case with Friedman-Halpern's algebraic 
plausibility measures. This assumption entails that Pl(yV) = ®pj^^y^r Pl{{-^}) fo'^ all W € 
even for W = ^ since Op is the identity for (Bp. Hence, defining 
Pl{{A}) for all complete assignments A of 5 suffices to describe PI. Moreover, in this 
case, the three conditions defining plausibility measures {Pl{dom{S)) = Ip, Pl{$) = Op, 
and {Wi C W2) ^ {Pl{Wi) <p Pl{W2))) are equivalent to just ©^^^^^^(5) Pl{{A]) = Ip, 
using the monotonicity of (Bp for the third condition. This means that we can deal with 
plausibility distributions instead of plausibility measures: 
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Definition 15. A plausibility distribution over S is a function Vs '■ dom{S) Ep such 

that ®pA&dom{S)'^s{A) = Ip. 

The normalization condition imposed on plausibility distributions is simply a gener- 
alization of the convention that probabilities sum up to 1. It captures the fact that the 
disjunction of all the assignments of S has Ip as a plausibility degree. 

Proposition 1. A plausibility distribution Vs can be extended to give a plausibility distri- 
bution Vs> over every S' C S, defined by Vs> = (Bpg^^i Vs- 

4.3 Feasibility Structure 

Feasibilities define whether a decision is possible or not, and are therefore expressed as 
booleans in {t, /}. This set is equipped with the total order <booi satisfying / -<booi t. 

Boolean scoped functions expressing feasibilities are combined using the operator A, 
since an assignment of decision variables is feasible iff all feasibility functions agree that 
this assignment is feasible. 

Given a scoped function Fj expressing feasibilities, we can compute whether an assign- 
ment A of a set S of variables is feasible according to Fi by computing V5c(Fi)-S F'i(A), since 
A is feasible according to Fi iff one of its extensions over sc{Fi) is feasible. This means that 
the projection of feasibility functions onto a smaller scope uses the elimination operator V. 

As a result, feasibilities are expressed using the feasibility structure Sf = {{t, /}, V, A). 
Sf is not only a commutative semiring, but also a plausibility structure. Therefore, all 
plausibility notions and properties apply to feasibility. We may therefore speak of feasi- 
bility distributions, and the normalization condition VsJ^s = t imposed on a feasibility 
distribution J^s over S means that at least one decision must be feasible. 

4.4 Utility Structure 

Utilities express preferences and can take various forms. Typically, utilities can be combined 
with +. But utilities can also model priorities combined using min. Also, when utilities 
represent hard requirements such as goals to be achieved or properties to be satisfied, they 
can be modeled as booleans combined using A. More generally, utility degrees are defined as 
elements of a set Eu equipped with a partial order :<u. Smaller utility degrees are associated 
with less preferred events. Utility degrees are combined using an operator which is 
assumed to be associative and commutative. This guarantees that combined utilities do 
not depend on the way combination is performed. We also assume that admits an 
identity G Eu, representing indifference. This ensures the existence of a default utility 
degree when there are no utility scoped functions. These properties are captured in the 
following notion of utility structure. 

Definition 16. (E'tj,®^) is a utility structure iff it is a commutative monoid and E^ is 
equipped with a partial order Elements of E^ are called utility degrees. 

Eu may have a minimum element _L„ representing unacceptable events and which will 
be an annihilator for 0u (the combination of any event with an unacceptable one must be 
unacceptable too). (S)u is also usually monotonic. But these properties are not necessary to 
establish the forthcoming results. 
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The distinction between plausibilities, feasibilities, and utilities is important and can 
be justified using algebraic arguments. Since ®p and (8)u may be different operators (for 
example, = x and = + in usual probabilities with additive utilities), we must 
distinguish plausibilities and utilities. It is also necessary to distinguish feasibilities from 
utilities or plausibilities. Indeed, imagine a simple card game involving two players Pi and 
P2, each having three cards: a jack J, a queen Q, and a king K. Pi must first play one 
card X G {J,Q,K}, then P2 must play a card y G {J,Q,K}, and last Pi must play a card 
z € {J, Q, K}. A rule forbids to play the same card consecutively (feasibility functions F^y : 
X ^ y and Fyz : y ^ z). The goal for Pi is that his two cards x and z have a value strictly 
better than P2's card y. By setting J < Q < K, this requirement corresponds to two utility 
functions Uxy : x > y and Uyz : z > y. In order to compute optimal decisions in presence of 
unfeasibilities, we must restrict optimizations (eliminations of decision variables with max 
or min) to feasible values: instead of max^; miuy maXz{Uxy A Uyz), we must compute: 



In the latter quantity, feasibility functions concerning P2's play (y) are taken into account 
using logical connective — >, so that P2's unfeasible decisions are ignored in the set of all 
scenarios considered. Feasibility functions concerning Pi's last move (z) are taken into ac- 
count using A, so that Pi does not consider scenarios in which he achieves a forbidden move. 
Therefore, feasibility functions cannot be handled simply by using the same combination 
operator as for utility functions: we need to dissociate what is unfeasible for all decision 
makers (unfeasibility is absolute) from what is unacceptable or required for one decision 
maker only (utility is relative), i.e. what a decision maker wants from what a decision maker 
can do. 

At a more general level, for example when Uxy and Uyz are soft requirements or when we 
do not know exactly in advance who controls which variable, the logical connectives A and 

cannot be used anymore. In order to ignore unfeasible values in decision variables elimi- 
nation, we use the truncation operator ★ introduced in Definition 6. In order to eliminate a 
variable x from a scoped function while ignoring unfeasibilities indicated by a feasibility 
function Pj, we simply perform the elimination of x on (Pj ★ ip) instead of 93. This maps 
unfeasibilities to value 0, which is ignored by elimination operators (see Definition 6). On 
the example above, if Uxy and Uyz were additive gains and costs, we would compute 



4.5 Combining Plausibilities and Utilities via Expected Utility 

To define expected utilities, plausibilities and utilities must be combined. Consider a 
situation where a utility Uj is obtained with a plausibility pi for all i G [l,-^]) with 




which, by setting f ~< t, is logically equivalent to 



max min I P, 

X y \ 




max min ( Fxy * max {Fyz * {Uxy + Uyz)) 
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Pi ©p • . . (BpPN = Ip- = ((pi, ui ),..., (pat, liTv)) is classically called a lottery (von Neu- 
mann & Morgenstern, 1944). When we speak of expected utility, we implicitly speak of the 
expected utility EU{C) of a lottery C. 

The standard way to combine plausibilities and utilities is to use the probabilistic ex- 
pected utility (von Neumann & Morgenstern, 1944) defining EU{C) as jv] (Pi ^ ^i)- it 
aggregates plausibilities and utilities using the combination operator iSipu = x and projects 
the aggregated information using the elimination operator = +. However, alternative 
definitions exist: 

• If plausibilities are possibilities, then EU{C) = minjg[x,Ar] iiiax(l — pj, Uj) with the 
possibilistic pessimistic expected utility (Dubois & Prade, 1995) (i.e. ®u = min and 
®pu '■ (P; u) max(l— p, u)) and EU{C) = maxjgji jy] min(p,, Uj) with the possibilistic 
optimistic expected utility (Dubois & Prade, 1995) (i.e. (Bu = max and (8)pu = min). 

• If plausibilities are K-rankings and utilities are positive integers (Giang & Shenoy, 
2000), then EU{C) = min^gji^^] (pj -|- Uj) (i.e. ©„ = min and = +)• 

To generalize these definitions of EU{C), we start from Chu-Halpern's work on gener- 
alized expected utility (Chu & Halpern, 2003a, 2003b). 

Chu-Halpern's structure Generalized expected utility is defined in an expectation do- 
main, which is a tuple {Ep, Eu, E'^,(Bu, 'Sipu) such that: (1) E'p is a set of plausibility degrees 
and Eu is a set of utility degrees; (2) ®p„ : Ep x E^ ^ E'^ combines plausibilities with 
utilities and satisfies Ip iSipu u = u; (3) (Bu '■ E'^ ^ E'^ ^ E'^ is a commutative and associative 
operator which can aggregate the information combined using ©p^- 

When a decision problem is additive, i.e. when, for all plausibility degrees pi,P2 associ- 
ated with disjoint events, (pi ©pP2) ©pu u = (pi ©p„ u) ©„ (p2 ©p^ u), the generic definition 
of the expected utility of a lottery is: 

EU{C)= ®u {pi®puUi). 
ie[i,7V] 

Classical expectation domains also satisfy additional properties such as "©„ is mono- 
tonic" and "Op 

©pu u — Ou , where is the identity of ©^ . 

Restriction of Chu-Halpern's structure for sequential decision making If we 

use ©pa : Ep X Eu ^ E'^ and ©„ : E[^ x E'^ ^ E[^ to compute expected utilities at 
the first decision step, then we need to introduce operators ©^^ : Ep x E'^ ^ E'^ and 
©^ : E" X E" — > E" to compute expected utilities at the second decision step. In the end, if 
there are T decision steps, we must define T operators ©p„ and T operators ©„. In order to 
avoid the definition of an algebraic structure that would depend on the number of decision 
steps, we take Eu = E'u and work with only one operator ©p„ : Ep x Eu Eu and one 
operator (Bu '■ Eu x Eu ^ Eu- 

As for plausibilities, and for the sake of the future algorithms, we restrict Chu-Halpern's 
expectation domains {Ep, Eu,Eu, ©«, ©pu) so that ffi„ and ©p^j become closed and generalize 
properties of the initial ©^ and ©p^. However, this closure is not sufficient to deal with 
sequential decision making, because Chu-Halpern's expected utility is designed for one-step 
decision processes only. This is why we introduce three additional axioms for ffi„ and ©p„: 
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• The first axiom is similar to a standard axiom for lotteries (von Neumann & Mor- 
genstern, 1944) defining compound lotteries. It states that if a lottery £2 involves a 
utility u with plausibility p2, and if one of the utilities of a lottery £1 is the expected 
utility of £2 with plausibility pi, then it is as if utility u had been obtained with 
plausibility pi ®pP2- This gives the axiom pi {p2 ^pu u) = {pi ®pP2) ®pu u. 

• We further require that distributes over ©„. To justify this point, assume that a 
lottery C = {{pi,ui), {p2,U2)) is obtained with plausibility p. Two different versions 
of the contribution of £ to the global utility degree can be derived: the first is p 0pu 
iiPi ^pu "1*1 ) ffiu {P2 ^puU2)), and the second, which uses compound lotteries, is ((p®p 
Pi) ®pu ui) ®u Hp ^p P2) ®pu U2)- We want these two quantities to be equal for all 
p,pi,P2,ui,U2. This can be shown to be equivalent to a simpler property p(8>pu (ni 
U2) = {p ^pu ui) (Bu {p ®pu U2), i.e. that ®pu distributes over ©„. 

• Finally, we assume that is right monotonic (i.e. {ui <u U2) — > {p ^pu ui :<u 
p ®pu U2))- This means that if an agent prefers (strictly or not) an event ev2 to 
another event evi, and if both events have the same plausibility degree p, then the 
contribution of ev2 to the global expected utility degree must not be lesser than the 
contribution of evi. 

These axioms define the notion of expected utility structure. 

Definition 17. Let {Ep,(Bp,(^p) be a plausibility structure and let [Eu-,®u) be a utility 
structure. {Ep, E^, ®u, 'S'pu) is an expected utility structure iff 

©M) ®pu) is 0, semimodule on {Ep, Q)p, 'S)p) (cf. Definition 12), 

• ®u is monotonic for and <S>pu is right monotonic for :<u ({ui M2) — ^ {p®puUi 

P®pu U2))- 

Many structures considered in the literature are instances of expected utility structures, 
as shown in Proposition 2. The results presented in the remaining of the article hold not only 
for these usual expected utility structures, but more generally for all structures satisfying 
the axioms specified in Definitions 14, 16, and 17. 

Proposition 2. The structures in Table 1 are expected utility structures. 

It is possible to define more complex expected utility structures from existing ones. For 
example, from two expected utility structures {Ep., Eu, ®pu) and {E'^., E'^, ©(^, ®p„), it is 
possible to build a compound expected utility structure {Ep x Ep, Eu x E'u, ®u, ®pu)- This 
can be used to deal simultaneously with probabilistic and possibilistic expected utility or 
more generally to deal with tuples of expected utilities. 

The business dinner example To fiesh out these definitions, we consider the following 
toy example, which will be referred to in the sequel. It does not correspond to a concrete 
real-life problem, but is used for its simplicity. Peter invites John and Mary (a divorced 
couple) to a business dinner in order to convince them to invest in his company. Peter 
knows that if John is present at the end of the dinner, he will invest SlQK. The same holds 
for Mary with $50K. Peter knows that John and Mary will not come together ( one of them 
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Table 1: Expected utility structures for: 1. probabilistic expected utility with additive 
utilities (allows the probabilistic expected utility of a cost or a gain to be com- 
puted); 2. probabilistic expected utility with multiplicative utilities, also called 
probabilistic expected satisfaction (allows the probability of satisfaction of some 
constraints to be computed); 3. possibilistic optimistic expected utility; 4. pos- 
sibilistic pessimistic expected utility; 5. qualitative utility with K-rankings and 
with only positive utilities; 6. boolean optimistic expected utility with conjunc- 
tive utilities (allows one to know whether there exists a possible world in which all 
goals of a set of goals G are satisfied); -^hooi denotes the order on booleans such 
that / -<hooi t; 7. boolean pessimistic expected utility with conjunctive utilities 
(allows one to know whether in all possible worlds, all goals of a set of goals G are 
satisfied); 8. boolean optimistic expected utility with disjunctive utilities (allows 
one to know whether there exists a possible world in which at least one goal of a 
set of goals G is satisfied); 9. boolean pessimistic expected utility with disjunctive 
utilities (allows one to know whether in all possible worlds, at least one goal of a 
set of goals G is satisfied). 



has to baby-sit their child), that at least one of them will come, and that the case "John 
comes and Mary does not" occurs with a probability of 0.6. As for the menu, Peter can order 
fish or meat for the main course, and white or red for the wine. However, the restaurant 
does not serve fish and red wine together. John does not like white wine and Mary does not 
like meat. If the menu does not suit them, they will leave the dinner. If John comes, Peter 
does not want him to leave the dinner because he is his best friend. 

Example. The dinner problem uses the expected utility structure representing probabilistic 
expected additive utility (row 1 in Table 1): the plausibility structure is (M"*", -|-, x), ®u = +> 
®pu = X; o-n-d utilities are additive gains ({Eu,<^u) = (1^ U { — oo}, +), with the convention 
that u + (— oo) = — oo ). 

4.6 Relation with Existing Structures 

If we compare the algebraic structures defined with existing ones (Friedman & Halpern, 
1995; Halpern, 2001; Chu & Halpern, 2003a), we can observe that: 



438 



The PFU Framework 



• The structures defined here are less general than Friedman-Chu-Halpern's, since ad- 
ditional axioms are introduced. For example, plausibility structures are not able to 
model belief functions (Shafer, 1976), which are not decomposable, whereas this is 
possible using Friedman-Halpern's plausibility measures (however, the authors are 
not aware of existing schemes for decision theory using belief functions). More- 
over, for one-step decision processes, Chu-Halpern's generalized expected utility is 
more general, since it assumes that <Sipu '■ Ep x —<■ E'^ whereas we consider 
^pu '■ Ep X Eu — > Ey_- 

• Conversely, the structures defined here can deal with multi-step decision processes 
whereas Chu-Halpern's generalized expected utility is designed for one-shot decision 
processes. Beyond this, other axioms, such as the use of closed operators, are essen- 
tially motivated by operational reasons. We use a less expressive structure for the 
sake of future algorithms (cf. Section 8). 

As a set Ep of plausibility degrees and a set Eu of utility degrees are defined, plausibil- 
ities and utilities must be cardinal. Purely ordinal approaches such as CP-nets (Boutilier, 
Brafman, Domshlak, Hoos, & Poole, 2004), which, like Bayesian networks, exploit the no- 
tion of conditional independence to express a network of purely ordinal preference relations, 
are not covered. 

As (^pu takes values in Eu, it is implicitly assumed that plausibilities and utilities are 
commensurable: works from Fargier and Perny (1999), describing a purely ordinal approach, 
where qualitative preferences and plausibilities are not necessarily commensurable, are not 
captured either. Also, works from Giang and Shenoy (2005), which satisfy all required 
associativity, commutativity, identity, annihilator, and distributivity properties, are not 
covered because they implicitly use iSipu ■ Ep x Eu ^ E'^ with Eu 7^ E'u (even if the 
expected utility EU{C) = {pi®puUi) ®u{P2®puU2) of a lottery C = ((pi,ui), {p2,U2)) stays 
in Eu). 

Furthermore, some axioms entail that only distributional plausibilities are covered (the 
plausibility of a set of variable assignments is determined by the plausibilities of each covered 
complete assignment): Dempster-Shafer belief functions (Shafer, 1976) or Choquet expected 
utility (Schmeidler, 1989) are not encompassed. Finally, as only one partial order on Eu 
is defined, it is assumed that the decision makers share the same preferences over utilities. 

4.7 Summary 

In this section, we have introduced expected utility structures, which are the first element 
of the PFU framework. They specify how plausibilities are combined and projected (using 
®p and ®p respectively), how utilities are combined (using (gj^), and how plausibilities and 
utilities are aggregated to define generalized expected utility (using and ®pu)- The 
structure chosen is inspired by Friedman-Chu-Halpern's plausibility measures and gener- 
alized expected utility. The main differences lie in the addition of axioms to deal with 
multi-step decision processes and in the use of extended domains to have closed operators, 
motivated by operational reasons. 
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5. Plausibility-Feasibility-Utility Networks 

The second element of the PFU framework is a network of scoped functions Pi, Fi, and 
Ui (cf. Equation 8) over a set of variables V. This network defines a compact and struc- 
tured representation of the state of the environment, of the decisions, and of the global 
plausibilities, feasibilities, and utilities which hold over them. 

In the rest of the article, a plausibility function denotes a scoped function to Ep (the 
set of plausibility degrees), a feasibility function is a scoped function to {t, f} (the set 
of feasibility degrees), and a utility function, a scoped function to (the set of utility 
degrees). 

5.1 Variables 

In structured representations, decisions are represented using decision variables, which are 
directly controlled by an agent, and the state of the environment is represented by envi- 
ronment variables, which are not directly controlled by an agent. The notion of agent used 
here is restricted to cooperative and adversarial decision makers (if there is an uncertainty 
on the way a decision maker behaves, then the decisions he controls will be modeled as 
environment variables) . We use Vd to denote the set of decision variables and Ve to denote 
the set of environment variables. Vd and Ve form a partition of V. 

Example. The dinner problem can be modeled using six variables: bpj and bpM (value t 
or f), representing John's and Mary's presence at the beginning, epj and epM (value t or 
f), representing their presence at the end, mc (value fish or meat), representing the main 
course choice, and w (value white or red), representing the wine choice. Thus, we have 
Vd = {mc, w} and Ve = {bpj, bpM, epj, epAi}. 

5.2 Decomposing Plausibilities and Feasibilities into Local Functions 

Using combined local functions to represent a global one raises some considerations: how 
and when such local functions can be obtained from a global one, and conversely, when 
such local functions are directly used, which implicit assumptions on the global function 
are made. We now show that all these questions boil down to the notion of conditional 
independence. In the following definitions and propositions, {Ep,(Bp,^p) corresponds to a 
plausibility structure. 

5.2.1 Preliminaries: Generalization of Bayesian Networks Results 

Assume that we want to express a global plausibility distribution Vs (cf- Definition 15) 
as a combination of local plausibility functions Pi. As work on Bayesian networks (Pearl, 
1988) has shown, the factorization of a joint distribution is essentially related to the no- 
tion of conditional independence. To introduce conditional independence, we first define 
conditional plausibility distributions. 

Definition 18. A plausibility distribution Vs over S is said to be conditionable iff there 
exists a set of functions denoted Vg^ \ S2 (^''^^ function for each pair Si, S2 of disjoint subsets 
of S) such that if 81,82, 83 are disjoint subsets of S, then 
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(a) for all assignments A of S2 such that "Psjl^) Op; ^Si|S2(^) ^-^ ^ plausibility distri- 
bution over Si,^ 

(b) Vs,\ib=Vs„ 

(C) ®p 3,^^3^,32 13; ='P32\Ss> 

(d) 'Psi ,52 I 53 = I 32 ,33 ®P ^52 I 53 ) 

( e) {Vs., 32,3, = Vs, I 53 ®P ^52 I 53 ®P ^53) ^ (^5i,S2 I 53 = | 53 ®V ^52 | 53)- 
■P5j 1^2 ^-^ called the conditional plausibility distribution oi Si given ^2. 

Condition (a) means that conditional plausibility distributions must be normalized. 
Condition (b) means that the information given by an empty set of variables does not 
change the plausibilities over the states of the environment. Condition (c) means that 
conditional plausibility distributions are consistent from the marginalization point of view. 
Condition (d) is the analog of the so-called chain rule with probabilities. Condition (e) is a 
kind of weak division axiom. ^ 

Proposition 3 gives simple conditions on a plausibility structure, satisfied in all usual 
frameworks, that suffice for plausibility distributions to be conditionable. 

Definition 19. A plausibility structure {Ep,(Bp,^p) is called a conditionable plausibility 
structure iff it satisfies the axioms: 

• if Pi P2 o,nd p2 / Op, then max{p ^ Ep\pi = p <SipP2} exists and is :<p Ip, 

• 'if Pi ^p P2, then there exists a unique p (z Ep such that Pi = p ®pP2, 

• if Pi ^p P2, then there exists a unique p (z Ep such that p2 = p ®pPi- 

Proposition 3. If {Ep^®p^®p) is a conditionable plausibility structure, then all plausibil- 
ity distributions are conditionable: it suffices to define Vs, \ S2 ^y Vs, \ S2 (^) ~ max{p € 
Ep I Vs,,32{^) =P®p 'Psii^)} for all A G dom{Si U ^2) satisfying Vsil^) / Op. 

The systematic definition of conditional plausibility distributions given in Proposition 3 
fits with the usual definitions of conditional distributions, which are, with probabilities, 
'T5,|52(^) = Vs„S2{A)/Vs2{Ay\ with K-rankings, ''Vs,\S2{A) = Vs„32{A) - Vs2{Ay\ 
and with possibility degrees combined using min, ^^Vs,\S2iA) = Vsi,32{A) if 'Psi,32{A) < 
Vs2 {A) , 1 otherwise" . In the following, every conditioning statement Vs, \ S2 condi- 
tionable plausibility structures will refer to the canonical notion of conditioning given in 
Proposition 3. Conditional independence can now be defined. 

3. To avoid specifying that properties of Vsi \ S2 hold only for assignments A of Si US2 satisfying 'Ps2{^) 7^ 
Op, we use expressions such as "Pg^ | s, = (fi" to denote "VA e dom{Si U S2), (Ps,(j4) / Op) — > 

(Psi|S,(^) = ^(^))". 

4. Compared to Friedman and Halpern's conditional plausibility measures (Friedman & Halpern, 1995; 
Halpern, 2001), (c) is the analog of axiom (Algl), (d) is the analog of axiom (Alg2), (e) is the analog of 
axiom (Algl), and axiom (Alg3) corresponds to the distributivity of (g)p over ©p. 
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Definition 20. Let (-Ep, ©p, <8>p) be a conditionable plausibility structure. Let Vs be a 
plausibility distribution over S and Si, 32,83 be disjoint subsets of S. Si is said to be 
conditionally independent of S2 given S3, denoted I{Si,S2 {S^), iff 'Psi,S2\s-j, = 'T^Si\s-a ®p 

^52 I S3 • 

This means that Si is conditionally independent of 52 given S3, iff the problem can be 
split into one part depending on Si and S3, and another part depending on ^2 and 53.^ 
This definition satisfies the usual properties of conditional independence, as Proposition 4 
shows. 

Proposition 4. /(., . | .) satisfies the semigraphoid axioms: 

1. symmetry: ^2 | 5*3) I{S2, Si \ S3), 

2. decomposition: I {Si, ^2 U ^3 | ^4) — > I {Si, S2 \ S/i), 

3. weak union: L{Si, 6*2 U | S/i) L{Si, ^2 | 6*3 U 5*4), 

4- contraction: (/(-Si, 5*2 | Si) A L{Si, 5*3 | 6*2 U 54)) — > I{Si, 5*2 U ^3 | S^). 

Proposition 4 makes it possible to use Bayesian network techniques to express informa- 
tion in a compact way. With Bayesian networks, a DAG of variables is used to represent 
conditional independences between the variables (Pearl, 1988). In some cases, such as image 
processing and statistical physics, it is more natural to express conditional independences 
between sets of variables. If probabilities are used, such situations can be modeled us- 
ing chain graphs (Frydenberg, 1990). In a chain graph, the DAG defined is not a DAG of 
variables, but a DAG of sets of variables, called components. Conditional probability distri- 
butions Px\paG{x) of variables are replaced by conditional probability distributions Pc\paG{c) 
of components, each Pc\paG{c) being expressed in a factored form (/j^ x (/?2 x • • • x V'fc • Markov 
random fields (Chellappa & Jain, 1993) correspond to the case in which there is a unique 
component equal to V, and in which the factored form of Py looks like \/Z x Y\j£j^^^ 
(Gibbs distribution). 

We now formally introduce DAGs over sets of variables, called DAGs of components, 
and then use them to factor plausibility distributions. 

Definition 21. A DAG G is said to be a DAG of components over a set of variables S iff 
the vertices of G form a partition of S. C{G) denotes the set of components of G. For each 
c € C{G), paG{c) denotes the set of variables included in the parents of c in G, and ndcic) 
denotes the set of variables included in the non- descendant components of c in G. 

Definition 22. Let {Ep,(Bp,0p) be a conditionable plausibility structure. Let Vs be a 
plausibility distribution over S and let G be a DAG of components over S. G is said to 
be compatible with Vs iff I{c,ndG{c) — paG{c) \ paG{c)) for all c G C{G) (c is conditionally 
independent of its non- descendants given its parents). 

5. Definition 20 differs from Halpern's, wfiicfi is "Si is conditionally independent (CI) of S2 given Ss iff 
Vs-^l 82,83 = I S3 S'Hd 1^S2\Si,S3 = ^32133"- Halpern (2001) called the definition we adopt non- 
interactivity (NI) and showed that NI is weaker than CI. This implies that NI is satisfied more often 
and may lead to more factorizations. Halpern gave a simple axiom (axiom (Alg4')) under which CI and 
NI are equivalent. Though this axiom holds in many usual frameworks, it does not hold with possibility 
degrees combined using min, a case covered by the PFU algebraic structure. 
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Theorem 1. (Conditional independence and factorization) Let {Ep,(Bp,^p) be a condition- 
able plausibility structure and let G be a DAG of components over S. 

(a) If G is compatible with a plausibility distribution Vs overS, thenVs — ®Pc£C{G)'^<^\pO'G' 

(b) If, for allc£ C{G), there is a function '^c,paQ{c) such that ip(.^pg_^(^(.^(^Aj is a plausibility 
distribution over c for all assignments A of padc), then 75 = '^pc£C{G) ^c,paG{c) ^■s o- 
plausibility distribution over S with which G is compatible. 

Theorem 1 Unks conditional independence and factorization. Theorem 1(a) is a gener- 
ahzation of the usual result of Bayesian networks (Pearl, 1988) which says that if a DAG 
of variables is compatible with a probability distribution Ps, then Ps can be factored as 
Ps = YlxeS -^^IpaGi^)' Theorem 1(b) is a generalization of the standard result of Bayesian 
networks (Pearl, 1988) which says that, given a DAG G of variables in S, if conditional 
probabilities Px\paG{x) defined for each variable x € 5, then Ylx&s ^x\paG(x) defines a 
probability distribution over S with which G is compatible. Both results are generalizations 
since they hold for arbitrary plausibility distributions (and not for probability distributions 
only). Results similar in spirit are provided by Halpern (2001), who gives some conditions 
under which a plausibility measure can be represented by a Bayesian network. 

Theorem 1(a) entails that, in order to factor a global plausibility distribution Vs, it 
suffices to define a DAG of components compatible with it, i.e. to express conditional 
independences. To define such a DAG, the following systematic procedure can be used. 
The initial DAG of components is an empty DAG G. While C{G) = {ci, . . . , Cfc_i} is not a 
partition of S, do: 

1. Let 5fc = ci U . . . U Cfc_i; choose a subset of the set 5 — 5^ of variables not already 
considered. 

2. Add component to G and find a minimal subset pok of 5^ such that I{ck, Sk — 
pa^ \pak). Add edges directed from components containing at least one variable in 
pak to Cfc, so that padck) = ^{cG{cl,...,Ck-l})/{cnpa^,J^ll))C■ 

The resulting DAG of components is guaranteed to be compatible with Vs, which im- 
plies, using Theorem 1(a), that the local functions Pi representing Vs can simply be defined 
as the functions in the set {V(.\paG{c)i'^ £ ^{G)}- In practice, if there is a reasonable notion 
of causes and effects, then networks that are smaller or somehow easier to build can be 
obtained by using the following two heuristics in order to choose at each step of the 
procedure above: 

(Rl) Consider causes before effects: in the dinner problem, this suggests not putting epj 
in Cfc if its causes hpj and w are not in Sk- 

(R2) Gather in a component variables that are correlated even when all variables in Sk are 
assigned: bpj and bpM are correlated and (Rl) does not apply. Indeed, we cannot 
say that bpj has a causal influence on bpM, or that bpM has a causal influence on bpj, 
since which of Mary or John chooses flrst if (s)he baby-sits is not specified. We can 
even assume that bpj and bpM are correlated via an unmodeled common cause, such 
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as a coin toss that determines the baby-sitter. Hence, bpj and bpM can be put in the 
same component c = {bpj, bpM}-^ 

We say that (Rl) and (R2) build a DAG respecting causahty. They must be seen just as 
possible mechanisms that help in identifying conditional independences by using the notions 
of causes and effects. 

All the previous results extending Bayesian networks results to plausibility distributions 
also apply to feasibilities. Indeed, the feasibility structure Sf = ({t, /}, V, A) is a particular 
case of a conditionable plausibility structure, since it satisfies the axioms of Definition 19. 
We may therefore speak of conditional feasibility distribution. If 5 is a set of decision 
variables, the construction of a DAG compatible with a feasibility distribution J^s leads to 
the factorization Ts = /\ceC{G) ^clpadc)- 

5.2.2 Taking the Differenty Types of Variables into Account 

The material defined in the previous subsection enables us to factor one plausibility distri- 
bution defined over the set Ve of environment variables and one feasibility distribution 
J-Vi) defined over the set Vo of decision variables. However, dealing with just one plausibility 
distribution over Ve and one feasibility distribution over Vd is not sufficient. 

Indeed, for plausibilities, decision variables can influence the environment (for example, 
the health state of a patient depends on the treatment chosen for him by a doctor). Rather 
than expressing one plausibility distribution over Ve, we want to express a family of plau- 
sibility distributions over Ve, one for each assignment of Vd- To make this clear, we define 
controlled plausibility distributions. 

Definition 23. A plausibility distribution over Ve controlled by Vn (or just a controlled 
plausibility distribution^, denoted 7've||Vd, is a function dom(VE U Vd) — > Ep, such that 
for all assignments Ad of Vd, Vy^ \\ Vd(^-d) is a plausibility distribution over Ve- 

For feasibilities, it goes the other way around: the values of environment variables can 
constrain the possible decisions (for example, an unmanned aerial vehicle which is flying 
cannot take off). Thus, we want to express a family of feasibility distributions over Vd, 
one for each assignment of Ve- In other words, we want to express a controlled feasibility 
distribution Ty^ 1 1 . 

In order to directly reuse Theorem 1 for controlled distributions, we introduce the notion 
of the completion of a controlled distribution. This allows us to extend a distribution to 
the full set of variables V by assigning the same plausibility (resp. feasibility) degree to 
every assignment of Vd (resp. Ve), and to work with only one plausibility (resp. feasibility) 
distribution. 

6. Components such as {bpj,fepj\/} could be broken by assuming for example that bpM causally influences 
bpj, i.e. that Mary chooses if she baby-sits first. We can (and prefer to) keep the component as 
{bpj, fop A/} because, in general, "breaking" components can increase the scopes of the functions involved. 
For example, assume that we want to model plausibilities over variables representing colors of pixels of 
an N X N image, such that the color of a pixel probabilistically depends on the colors of its 4 neighbors 
only. With a component approach, results of Markov random fields (Chellappa & Jain, 1993) show that 
the local functions obtained have scopes of size 5 only, whereas with a component-breaking mechanism, 
the size of the largest scope is linear in A^. 
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Proposition 5. Let (-Bp, ©p, i^p) be a conditionable plausibility structure. Then, for all 
n S N*, there exists a unique po such that ©pjgji n}Po — ^p- 

Definition 24. Let {Ep, (Bp, ©p) be a conditionable plausibility structure and let Vy^ || be 
a controlled plausibility distribution. Then, the completion of Vy^ \ \Vd ^-^ ^ function denoted 
Vve,Vd' defined by Vve,Vd — '^Ve \ \Vd ®pVOj where po is the unique element of Ep such that 
®Pie[i \dom{VD)\]P^ ~ (^^^ cardinality of a set T is denoted \V\). 

In other words, 'Pve,Vd is defined from Vy^ \ \y^ by assigning the same plausibility degree 
Pq to all assignments of Vd- In the case of probability theory, it corresponds to saying that 
the assignments of Vd are equiprobable. The definition of the completion of a controlled 
plausibility distribution could be made more flexible: instead of defining a uniform plausibil- 
ity distribution over Vq, we could define a plausibility distribution such that no assignment 
of Vd has Op as a plausibility degree. We arbitrarily choose the uniform distribution, the 
goal being just to introduce some prior plausibilities over decision variables, for the sake of 
factorization. 

Proposition 6. Let Vve,Vo completion of a controlled plausibility distribution Vy^ \ \y^ 

Then, VyE,VD plausibility distribution over Ve U Vd and Vy^ \ = Vy^ \ \Vd- 

As a result, we use 'Pyg | y^ to denote Vy^wy^ (and this is equivalent). Similarly, it is 
possible to complete a controlled feasibility distribution .Fy^ \ \Ve- 

5.2.3 A First Factorization 

Proposition 7 below, entailed by Theorem 1(a), shows how to obtain a first factorization of 
'^Ve\Vd and J^y^jy^. 

Definition 25. A DAG G is a typed DAG of components over Ve U Vd iff the vertices of 
G form a partition of Ve U Vd such that each element of this partition is a subset of either 
Vd or Ve- Each vertex of G is called a component. The set of components contained in 
Ve (environment components) is denoted Ce{G) and the set of components contained in Vd 
(decision components) is denoted Cd{G). 

Proposition 7. Let G be a typed DAG of components over Ve U Vd- Let Gp be the partial 
graph of G induced by the arcs of G incident to environment components. Let Gj be the 
partial graph of G induced by the arcs of G incident to decision components. Lf Gp is 
compatible with the completion of Vy^\\y^ (cf. Definition 22) and Gf is compatible with 
the completion of J^y^^^y^, then 

'PVe \Vd= ®P I pacic) and Ty^ I y^ = A Tc I pacic) ■ 

c£Ce{G) ceCoiO] 

This allows us to specify local Pi and Fi functions: it suffices to express each Vcipaalc) 
and each J^c\paa(c) to express Vy^ | y^ and J^y^ | y^ in a compact way. In fact, we could have 
defined two DAGs, one for the factorization of Vy^ | y^ and the other for the factorization 
of J-yjy^y^, but these two DAGs can actually always be merged as soon as we make the 
(undemanding) assumption that it is impossible, given x € Vd and y £ Ve, that both x 
influences y and y constrains the possible decision values for x. This assumption ensures 
that the union of the two DAGs does not create cycles. We use just one DAG for simplicity. 
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Example. Consider the dinner problem to illustrate the first factorization step. One way 
to obtain G is to use the causality-based reasoning described after Theorem 1. We start 
with an empty DAG. As epj and epM are both effects of bpj, bpM, w, or mc, they are not 
considered in the first component ci. bpj can be chosen as a variable to add to ci, because 
we cannot say that bpj is necessarily an effect of another variable. As previously explained, 
bpj can be a cause ofbpM, or an effect ofbpM, or bpj may be correlated with bpM via an 
unmodeled cause. As a result, we get c\ = {bpj,bpM} as a first component. Obviously, c\ 
has no parents in the DAG because it is the first added component. 

Then, as epj and epM are effects of w or mc, we do not consider epj or epM in the 
second component C2. Since w is not necessarily an effect of mc, we can add w to C2. The 
dinner problem specifies that ordering fish and red wine simultaneously is not feasible, but 
we do not know whether the wine is chosen before or after the main course, i.e. w can 
be a cause or an effect of mc. As a result, we take C2 = {mc,w}. As the menu choice is 
independent from who is present at the beginning, ci has no parent in the temporary DA G. 

As epj is a direct effect of bpj and w only (John leaves the dinner if white wine is 
chosen), we can add epj to C3. Moreover, epj is not correlated with epM when ci U C2 = 
{bpj,bpM,mc,w} is assigned. Therefore, we take C3 = {epj}. Given that epj depends both 
on bpj and w, C3 gets {bpj, bpM} and {mc, w} as parents. Finally, C4 = {cpm}, and as epu 
depends on bpM and mc (Mary leaves if meat is chosen) and is independent of epj given bpM 
and mc, we have that I{{epM}, {^Pj, bpj, w} \ {bpM,mc}). This entails that {cpm} is added 
to the DAG with {bpj, bpM} and {mc, w} as parents. Therefore, we get Cd{G) = {{mc, w}} 
as the set of decision components and Ce{G) = {{bpj,bpM},{epj},{epM}} as the set of 
environment components. The DAG of components is shown in Figure la. 

Using Proposition 7, we know that the joint probability distribution factors as Vy^ \ Vd ~ 
'Pbpj,bpM X 'Pepj\bpj,bpM,mc,w X 'Pep^ \bpj ,bpM ,mc,w and that the joint feasibility distribution 
can be factored as Ty^ \ = Tmc,w ■ 

5.2.4 Further Factorization Steps 

Proposition 7 provides us with a decomposition of Vy^ \ Vd ^Vd \ Ve based on the condi- 
tional independence relation /(., . | .) of Definition 20. It may be possible to perform further 
factorization steps by factoring each Vc\pac{c) ^ set of local plausibility functions Pi and 
factoring each J'c\paG(c) a set of local feasibility functions Fi. 

• In some cases, expressing factors of Vcipadc) or J^dpadc) is quite natural. For exam- 
ple, if (8)p = A, if variables in an environment component c = {xij\i,j £ [1)?^]} 
without parents represent pixel colors, and if we want to model in Vc that two 
adjacent pixels have different colors, it is natural to define a set of binary differ- 
ence constraints Sa:ij,Xk,i and to factor Vc as Vc = {/\i(z[i^n-i] '^j&li,n]^xi,j,xi+i,j) A 
(Ajg[i,n] Ajg[i^„_i] j+i) ■ Such a decomposition cannot be obtained based only 
on the conditional independence relation /(., . | .) of Definition 20. 

• In some settings, as in Markov random fields (Chellappa & Jain, 1993), systematic 
techniques exist to obtain such factorizations. The Bayesian network community 
also offers systematic techniques: with hybrid networks (Dechter & Larkin, 2001), 
we can extract the deterministic information contained in conditional probability 
distributions. More precisely, a conditional probability distribution Px\paG{x) can 
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be expressed as Pxlpadx) = Px\pao{x) ^ T, where F is the 0-1 function defined by 

= { 1 tthevwSt^^^^ ~ ° factorization of Pxlpadx) as Pxlpadx) x T can 

be computationally relevant when constraint propagation techniques on 0-1 functions 
are used to solve hybrid networks. 

• We may use another weaker definition of conditional independence: in valuation-based 
systems (Shenoy, 1994), Si and 5*2 are said to be conditionally independent given 53 
with regard to a function ^Si,S2,Ss if this function factors into two scoped functions 
with scopes Si U ^3 and ^2 U 53. This definition is not used for the first factorization 
step because it destroys the normalization conditions which may be useful from a 
computational point of view. 

These additional factorization steps are of interest because decreasing the size of the 
scopes of the functions involved or adding redundant information in the problem can be 
computationally useful. 

For every environment component c, if "P^ € Fact(c)" stands for "Pj is a factor of 
Vclpaaic)" 1 the second factorization gives us 

T^c\pac{c) ~ ®p Pi- 
Pi£Fact{c) 

As ®p^'Pc\pac{c) = Ip? the Pi functions in Fact{c) satisfy the normalization condition 
®pJy'^pPi(iFact{c) ^i) = Their scopes sc{Pi) are contained in sc{Vc\paG{c)) = cUpacic). 

For every decision component c, if "Fj G Fact{c)" stands for "Fj is a factor oi J^c\paGic)\ 
the second factorization gives us 

2\ panic) — „ /}• , Pi- 

Given that Vc^c|paG(c) = ^) the Fi functions in Fact{c) satisfy the normalization condition 
Vc {f\Fi(iFact{c) Pi) = i- Moreover, sc{Fi) C cUpadc). 

Other factorizations, which do not decrease the scopes of the functions involved, could 
also be exploited. Indeed, each scoped function Pi or Fi can itself have an internal local 
structure, as for instance when Pi is a noisy-OR gate (Pearl, 1988) in a Bayesian network, 
or in presence of context-specific independence (Boutilier, Friedman, Goldszmidt, & Koller, 
1996). Such internal local structures can be made explicit by representing functions with 
tools such as Algebraic Decision Diagrams (Bahar, Frohm, Gaona, Hachtel, Macii, Pardo, 
& Somenzi, 1993). In the rest of the article, we do not make any assumption on the way 
each scoped function is represented. 

Example. VbpjfipM can be expressed in terms of a first plausibility function Pi spec- 
ifying the probability of John and Mary being present at the beginning. Pi is defined 
by Pi{{bpj,t).{bpMj)) = 0.6, Pi{{bpjJ).{bpM,t)) = 0.4, and Pi{{bpj,t).{bpM,t)) = 
Pi{{bpj, f).{bpM, /)) = 0. We can also add redundant deterministic information with a sec- 
ond plausibility function P2 defined as the constraint bpj ^ bpM (P2{A) = 1 if the constraint 
is satisfied, otherwise). We get V^pj^p^^j = Pi ®p P2 o-nd Fact{{bpj,bpj^f}) = {Pi,P2}. 
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'^epj\bpj,bpM,mc,w ca?^ be Specified as a combination of two plausibility functions P3 and 
Pi. Pi expresses that if John is absent at the beginning, he is absent at the end: P3 is the 
hard constraint {bpj = f) —>■ {ep j = f) (P3{A) = 1 if the constraint is satisfied, otherwise). 
Then, P4 : {bpj = t) —i- {{epj = t) {w ^ white)) is a hard constraint specifying that 
John leaves iff white wine is chosen. Hence, we have Vepj\bpj,bpM,mc.w = -^3 ®p ^''t-^ 
Fact{{epj}) = {P3,P4}. Similarly, V^pm \bpj ,bpj^j ,mc,w — ^5 '^p -^6; with P^ , Pq defined as 
constraints, and Fact{{epM}) = {-Psi-Pe}- 

As for feasibilities, J-mc,w can be specified by a feasibility function Fi expressing that or- 
dering fish with red wine is not allowed: Fi : -i((mc = fish)f\{w = red)) and Fact{{mc, w}) 
= {Fi}. The association of local functions with components appears in Figure la. 

5.3 Local Utilities 

Local utilities can be defined over the states of the environment only (as in the utility of 
the health state of a patient), over decisions only (as in the utility of the decision of buying 
a car or not), or over the states of the environment and decisions (as in the utility of the 
result of a horse race and a bet on the race)/ 

In order to specify local utilities, one standard approach, used in CSPs and influence 
diagrams, is to directly define a set U of local utility functions, modeling preferences or 
hard requirements, over decision and environment variables. This set implicitly defines a 
global utility Uy = ®uUi&u Ui over all variables. If this factored form is obtained from 
a global joint utility, one may rely, when = +> on the work of Fishburn (1982) and 
Bacchus-Grove (1995), who introduced a notion of conditional independence for utilities. 

No normalization condition is imposed on local utilities. 

Example. In the dinner problem, three local utility functions can be defined. A binary 
utility function Ui expresses that Peter does not want John to leave the dinner: Ui is 
the hard constraint (bpj = t) — > (epj = t) (Ui{A) = if the constraint is satisfied, —oo 
otherwise). Two unary utility functions U2 and U3 over epj and epM respectively express the 
gains expected from the presences at the end: U2{{epj,t)) = 10 and U2{{epj, f)) = (John 
invests $10K if he is present at the end), while U-i{{epM-,'t)) = 50 and U-i{{epM-, f)) = 
(Mary invests $50K if she is present at the end). U2 and U3 can be viewed as soft constraints. 
All the local functions are represented in a graphical model in Figure lb. 

5.4 Formal Definition of PFU Networks 

We can now formally define Plausibility- Feasibility- Utility networks. The definition is jus- 
tified by the previous construction process, but it holds even if the plausibility structure is 
not conditionable. 



7. In influence diagrams, special nodes called value nodes are introduced to represent the outcome of 
decisions, and one utility function is associated with each of these value nodes (the utility of the outcome). 
In the PFU framework, we directly represent such utility functions as scoped functions which hold on 
the parents of value nodes. This explicitly express that utility functions are scoped functions, just as 
plausibility and feasibility functions. In other words, utility functions are directly utilities of the outcome 
of decision and environment variables assignments. 
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Figure 1: (a) DAG of components (b) Network of scoped functions. 



Definition 26. A Plausibility-Feasibility-Utility network on an expected utility structure 
is a tuple J\f = {V, G, P, F, U) such that the following conditions hold: 

• V = {xi,X2, . . .} is a finite set of finite domain variables. V is partitioned into Vd 
(decision variables) andVs (environment variables); 

• G is a typed DAG of components over Ve U Vd (cf. Definition 25); 

• P = {Pi,i-2i---} is a finite set of plausibility functions. Each Pi ^ P is associ- 
ated with a unique component c € Ce{G) such that sc{Pi) C cUpaG'(c). The set of 
Pi £ P associated with a component c G Ce{G) is denoted Fact{c) and must satisfy 

c 

• F = {Fi,i<2, . . .} is a finite set of feasibility functions. Each function Fi is associated 
with a unique component c G Cd{G) such that sc{Fi) C c U padc). The set of 
Fi £ F associated with a component c € Cd{G) is denoted Fact{c) and must satisfy 

V {^F,eFact{c)Fi) = t; 

• U = {Ui,U2, ■ ■ ■} is a finite set of utility functions. 
5.5 From PFU Networks to Global Functions 

We have seen how to obtain a PFU network expressing a global controlled plausibility 
distribution Vy^ \ \Vd' ^ global controlled feasibility distribution 1 1 , and a global utility 
Uy. Conversely, let M = {V, G, P, F, U) be a PFU network, i.e. a set of variables, a typed 
DAG of components, and sets of scoped functions. Then 

• the global function ^' = ®pp.^pPi is a controlled plausibility distribution of Ve given 
Vd- Moreover, by Theorem 1(b), if the plausibility structure is conditionable and if 
Gp is the partial DAG of G induced by the arcs incident to environment components, 
then Gp is compatible with the completion of 

• the global function $ = ApieP Fi is a controlled feasibility distribution of Vd given 
Ve- Moreover, by Theorem 1(b), if G/ is the partial DAG of G induced by the arcs of 
G incident to decision components, then Gf is compatible with the completion of 
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• fj- = ®uu,&u Ui is necessarily a global utility. 

We can therefore denote ^' by Vy^ || v^, ^ by J^y^ [[ y^, and fi by Uy- 

5.6 Back to Existing Frameworks 

Let us consider the formalisms described in Section 3 again. 

• A CSP (hard or soft) can easily be represented as a PFU network Af = {V, G, 0, 0, U): 
all variables in V are decision variables, G is reduced to a single decision component 
containing all variables, and constraints are represented by utility functions. Using 
feasibility functions to represent constraints, it would be impossible to represent in- 
consistent networks because of the normalization conditions on feasibilities. SAT is 
modeled similarly; the only difference is that constraints are replaced by clauses. 

• The same PFU network as above is used to represent the local functions of a quantified 
boolean formula or of a quantified CSP. The differences with CSPs or SAT appear 
when we consider queries on the network (see Section 6). 

• A Bayesian network can be modeled as TV = (y, G, P, 0,0): all variables in V are 
environment variables, G is the DAG of the BN, and P = {Px\paG{x)7^ ^ There 
is no feasibility or utility function. A chain graph is also modeled as J\f = {V, G, P, 0, 0), 
with G the DAG of components of the chain graph and P the set of factors of each 

Pclpacic)- 

• A stochastic CSP is represented by a PFU network TV = {V,G, P,^,U), where V 
is partitioned into Vd, the set of decision variables, and Ve, the set of stochastic 
variables, G is a DAG which depends on the relations between the stochastic variables, 
P is the set of probability distributions over the stochastic variables, and U is the set 
of constraints. 

• An influence diagram can be modeled by TV = {V, G, P, 0, U) such that Vd contains 
the decision variables, Ve contains the chance variables, G is the DAG of the influence 
diagram without the utility nodes and with arcs into random variables only (i.e. we 
keep only the so-called influence arcs), and P = {Px\paG{x)>x G Ve}- There are no 
feasibilities, and one utility function Ui is defined per utility variable u, the scope 
of Ui being padu). To represent valuation networks, a set F of feasibility functions 
is added. Note that the business dinner example could not have been modeled us- 
ing a standard influence diagram, since influence diagrams cannot model feasibilities 
(suitable extensions exist however, Shenoy, 2000). 

• A finite horizon probabilistic MDP can be modeled as = {V, G, P, F, U). If there are 
r time-steps, then Vd = {dt,t £ [l,r]}u{si} and Ve = {st,t G [2,T]};^ G is a DAG of 
components such that (a) each component contains one variable, (b) the unique parent 
of a decision component {dt} is {st}, (c) the parents of an environment component 
{st+i} are {st} and {dj; P = t G [1, T - 1]}, F = {F^, , t € [1, T]}, and 
U = {Rst,dt^^ ^ [li^]}- Modeling a finite horizon possibilistic MDP is similar. 

8. As there is no plausibility distribution over the initial state Si, si is not viewed as an environment 
variable. This corresponds to the special case where decision variables model problem parameters. 
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5.7 Summary 

In this section, we have introduced the second element of the PFU framework: a network of 
variables linked by local plausibility, feasibility, and utility functions, with a DAG capturing 
normalization conditions. The factorization of global plausibilities, feasibilities, and utilities 
into scoped functions was linked to conditional independence. 

6. Queries on a PFU Network 

A query will correspond to a reasoning task on the information expressed by a PFU network. 
If decision variables are involved in the PFU network considered, answering a query may 
provide decision rules. Examples of informal queries about the dinner problem are 

1. "What is the best menu choice if Peter does not know who is present at the beginning?" 

2. "What is the best menu choice if Peter knows who is present at the beginning?" 

3. "How should we maximize the expected investment if the restaurant chooses the main 
course first and Peter is pessimistic about this choice, then who is present at the 
beginning is observed, and last Peter chooses the wine?" 

Dissociating PFU networks from queries is consistent with the trend in the influence di- 
agram community to relax the so-called information links, as in Unconstrained Influence 
Diagrams (Jensen &: Vomlelova, 2002) or Limited Memory Influence Diagrams (Lauritzen 
& Nilsson, 2001): it explains the intuition that queries do not change the local relations 
between variables. 

In this section, we define a simple class of queries on PFU networks. We assume that 
a sequence of decisions must be performed, and that the order in which decisions and 
observations are made is known. We also make a no-forgetting assumption, that is, when 
making a decision, an agent is aware of all previous decisions and observations. From 
now on, the set of utility degrees Eu is assumed to be totally ordered. This total order 
assumption, which holds in most of the standard frameworks, implies that there always 
exists an optimal decision rule. See Subsection 6.7 for a discussion of how to extend the 
results to a partial order. 

Two definitions of the answer to a query are given, the first based on decision trees, 
and the second more operational. An equivalence between these two definitions is then 
established. 

6.1 Query Definition 

In order to formulate reasoning tasks on a PFU network, we use a sequence Sov of operator- 
variable(s) pairs. This sequence captures different aspects of the query: 

• Partial observabilities: Sov specifies the order in which decisions are made and envi- 
ronment variables are observed. If x S Ve appears to the left of y S Vo (for example 
Sov = . . . (©n, {x}) ■ ■ ■ (max, {y}) ■ ■ ■), this means that the value of x is known (ob- 
served) when a value for y is chosen. Conversely, Sov = . . . (max, {y}) . . . (©„, {x}) . . . 
if X is not observed when choosing y. 
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• Optimistic/pessimistic attitude concerning the decision makers: (max, {y}) is inserted 
in the ehmination sequence if the decision maker is optimistic about the behavior of 
the agent controlhng a decision variable y (i.e. if this agent is cooperative), and 
(min, {?/}) if one is pessimistic (i.e. if the agent controlhng y is an adversary). The 
operator used for environment variables will always be ©„, to model that expected 
utilities are sought.^ 

• Parameters of the decision making problem: the set S of variables that are not involved 
in Sov are kind of parameters. Their absence indicates that we want to obtain optimal 
expected utilities and/or optimal policies for each assignment of S. This is useful in 
order to evaluate several scenarios simultaneously. 

Example. The sequence corresponding to the informal query: "How should we maximize the 
expected investment if the restaurant chooses the main course first and Peter is pessimistic 
about this choice, then those present at the beginning of the dinner are observed, and last 
Peter chooses the wine before knowing who is present at the end?" is 

Sov = (min, {?nc}).(©„, {bpj, &pA/})-(max, {w}).(®„, {epj, epAi})- 

It models the fact that: (1) Peter is pessimistic about the main course (mm overmc), which 
is chosen without observing any variable (no variable to the left of mc in Sov); (2) Peter 
makes the best choice of wine (max over w) after the main course has been chosen and 
after knowing who is present at the beginning (w appears to the right of mc, bpj, and bpM 
in Sov), but before knowing who is present at the end (w appears to the left of epJ,epM)■ 
Specifically, bp J and bpM are partially observable, whereas epj and epM are unobservable. 
If the query becomes "What should Peter do if he observes those present at the beginning 
of the dinner and then chooses the wine before knowing who is present at the end?", then 
the sequence to use is Sov = (©„, 6pA/}). (max, {?/;}).(©„, {epj, cpm})- In this case, 
variable mc does not appear in the sequence anymore, which means that mc is a parameter 
and that an answer for each value of this parameter is sought. 

Definition 27. A query on a PFU network is a pair Q = {J\f, Sov) where M is a PFU 
network and Sov = {opi, Si) • {op2, S2) • ■ ■ {opk, Sk) is a sequence of operator-set of variables 
pairs such that 

(1) all the Si are disjoint; 

(2) either Si C V/j and opi = min or max, or Si Q Ve and opi = ©„; 

(3) variables not involved in any of the Si, called free variables, are decision variables; 

(4) for all variables x,y of different types (one is a decision variable, the other is an 
environment variable), if there is a directed path from the component which contains 
X to the component which contains y in the DAG of the PFU network Af, then x does 
not appear to the right of y in Sov, i.e. either x appears to the left ofy, or x is a free 
variable. 

9. If a decision is made by nature and if we have a plausibility distribution on that decision, then this 
decision will be viewed as an environment variable. 
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Condition (1) ensures that each variable is eliminated at most once. Condition (2) means 
that optimal decisions are sought for decision variables (either maximized if the decision 
maker who controls a decision variable is cooperative, or minimized if he is adversarial), 
whereas expected utilities are sought for environment variables. Condition (3) means that 
variables which are not eliminated in Sov act as problem parameters and are viewed as 
decision variables. Condition (4) means that if x and y are of different types and x is an 
ancestor of y, then x is assigned before y. This ensures that causality is respected for vari- 
ables of different types: for example, (A^, bpM,epj, epM})-(niax, {mc, w})), which 
violates condition (4), violates causality since the menu cannot be chosen after knowing who 
is present at the end. 

Variables appearing in Sov are called quantified variables, by analogy with quantified 
boolean formulas. The set of free variables is denoted by Vf^- Notice that the definition of 
queries does not prevent an environment variable from being "quantified" by min or max, 
because we may have (Bu = min or (Bu = max. Note also that it is straightforward that for 
every PFU network M, there exists at least one query on J\f without free variables. 

For alH G [1, fc], we define 

• the set l{Si) of variables appearing in Vjr or to the left of Si in Sov by l{Si) = 
y/.U(U,e[i,,„i]5,); 

• the set r{Si) of variables appearing to the right of Si in Sov by r(Si) = Ujg[j_|_i ^jSj. 
6.2 Semantic Answer to a Query 

In this subsection, we assume that the plausibility structure is conditionable (cf. Defini- 
tion 19). The controlled plausibility distribution T'v^wvd ~ ®pp.^pPi can then be com- 
pleted (cf. Definition 24) to give a plausibility distribution 'Pve,Vd over Ve^Vd- Similarly, 
the controlled feasibility distribution J^Vd\\Ve ~ ^Fi&F Fi can be completed to give a fea- 
sibility distribution J^Ve,Vd o^cr Ve UVd- We also use the global utility Uy = ®uUi & 
defined by the PFU network. 

Imagine that we want to answer the query Q = (M, Sov), where M is the network of 
the dinner problem and Sov = (min, {mc}).{®u, {bpj, 6pA/}).(max, {w}).{(Bu, {epj, epAi})- 

To answer such a query, we can use a decision tree. First, the restaurant chooses the 
worst possible main course, taking into account the feasibility distribution of mc. Here, 
J^mc{{iT^c, meat)) = J^rnc,wi{n^c, meat). {w, white)) \/ Trnc,w{{jnc, meat). {w, red)) = t y t = 
t. Similarly, !Frnc{{fnc, f ish)) = t. Both choices are feasible. Then, if A\ denotes the 
assignment of mc, the uncertainty over those present at the beginning given the main 
course choice is described by the probability distribution Vbpj^bpj^j | md^i)- For each possible 
assignment A2 of {bpj,bpM}, i-e. for each A2 such that Vi^j^hpM \mc{^i-^2) Op; Peter 
chooses the best wine while taking into account the feasibility J^w\mc,bpj,bpMi^'i^-^2)- if 
the restaurant chooses meat, Peter chooses an optimal value between red and white, and 
if the restaurant chooses fish, Peter can choose white wine only. Then, for each feasible 
assignment ^3 of w, the uncertainty regarding the presence of John and Mary at the end 
of the dinner is given by V^pj^epM \ bpj,bpM,mc,w 

{A1.A2.A3). 

Note that the conditional probabilities used in the decision tree above are not directly 
defined by the network. They must be computed from the global distributions. This 
computation can be a challenge in large problems. 
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The utility UviA1.A2.A3.A4) can be associated with each possible complete assignment 
A1.A2.A2.Ai of the variables. For each possible assignment ^1.^2-^3 of {bpj,bpM,mc,w}, 
the last stage, i.e. the one in which epj and epM are assigned, can be seen as a lottery (von 
Neumann & Morgenstern, 1944) whose expected utility is Y^A4edomi{epj ,epM}) ^^(^4) x ^(^4), 
where p{Ai) = Vepj,epM\bpj,bpM,mc,w{Ai.A2.As.Ai) and ^(^4) = i^y(Ai.^2-^3-^4)- This 
expected utility becomes the reward of the scenario over {bpM,bpj,mc,w} described by 
A1.A2.A3. It provides us with a criterion for choosing an optimal value for w. The step 
in which bp j and bpM are assigned can then be seen as a lottery, which provides us with a 
criterion for choosing a worst value for mc. The computation associated with the previously 
described process is: 



mm 



E 



'PbpjfipM I mc{Al ■A2) X 

A2&dom{{bpj,bpM}),'Pl,pj^bpj^j | rnc 



max 

A3£dom{w),:F^ I mc,bpj,bpj^ti^i-M-A3)=t 



( ^ ^ 'PepJ,ep^f \bpJ,bp^f,mc,w{Al.A2.A3.Ai^ 



X 



A, e dom({epj ep,,}) Z^y (Ai . ^2 -^3 -^4)) ) ) • 

l^epj,epM \bpj,bpM,mc.w(Al-A2.A3.A4) ^ '''' 

Decision rules for the decision variables (argmin and argmax) can be recorded during 
the computation. This formulation represents the decision process as a decision tree in 
which each internal level corresponds to variable assignments. Arcs associated with the 
assignment of a set of decision variables are weighted by the feasibility of the decision given 
the previous assignments. Arcs associated with the assignment of environment variables are 
weighted by the plausibility degree of the assignment given the previous assignments. Leaf 
nodes correspond to the utilities of complete assignments, and a node collects the values of 
its children to compute its own value. 

6.2.1 Formalization of the Decision Tree Procedure 

In order to formalize the decision tree procedure, some technical results are first introduced 
in Proposition 8. These results and the definitions preceding them can be skipped for a first 
reading. 

Definition 28. Let Vg^ \ be the conditional plausibility distribution of Si given S2 and let 
A € dom{S2). The function Vs^ |52(^) •'^^'^ well-defined iffVs2{A) 7^ Op. Similarly, 
if J^Si \S2 conditional feasibility distribution of Si given S2, then, for all A € dom{S2), 

J^Si |52(^) •^'^^^ well-defined iff Ts2[A) = t. 

Next, the conditioning can be defined directly for controlled plausibility distributions 
because for all A € domiyo)^ Vy^ ||y^(A) is a plausibility distribution over Ve'- 

Definition 29. Assume that the plausibility structure used is conditionable. Let 'Pve\\Vd 
be a controlled plausibility distribution and S,S' be two disjoint subsets ofVE- We define 
conditional controlled plausibility distributions by: for all A € dom{S U S" U Vd) such that 

'Ps'WVoi^) + T^s\s'\\Vd^A) = max{p G Ep\Vs,s' \\Vd{^) = P ^v'Ps' wvoi"^)}' 

the canonical definition of conditioning given in Proposition 3. Given a controlled feasi- 
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bility distribution Ty^\\v^, the definition of conditional controlled feasibility distributions 
^S\S'\\Ve f'^''" '5,5" disjoint subsets ofVo is similar. 

Proposition 8. Assume that the plausibility structure used is conditionable. Let Q = 
{Af, Sov) be a query where Sov = {opi, Si) • {op2, S2) ■ ■ ■ (opk, S^)- Let Vjr denote the set of 
free variables of Q. 

(1) If Si C Ve andVs.\i(^s.^{A) is well-defined, then there exists at least one A' € dom[Si) 
satisfying Vs^\i{s,){^-^') + %■ 

(2) If Si C Vd and J-s^ |;(5.)(j4) is well-defined, then there exists at least one A' S dom(Si) 
satisfying J='s^\i(_s,){^-A') = ^■ 

(3) IfVE^^ and Si is the leftmost set of environment variables appearing in Sov, then, 
for all A E dom{l{Si)), "Pg. | is well-defined. 

(4) If i,j G [1; fc]; i < j, Si C Ve, Sj C Ve, '''{Si) n l{Sj) C Vd (Sj is the first set of envi- 
ronment variables appearing to the right of Si in Sov), (A, A') S dom{l{Si))xdom{Si), 
'PSi \ i(Si)i^) well-defined, and \i{Si)i-^--^') Op; then, for all A" extending A. A' 
overl{Sj), \ i{Sj)i^") well-defined. 

(5) Ifi,j G [l,k], i < J, S^ C Vd, Sj C Vd, r{Si) n C Ve (Sj is the first set of 
decision variables appearing to the right of Si in Sov), {A, A') G dom{l{Si)) x dom{Si) , 
•^Si\i{Si){-^) well-defined, and J-s^\i[Si){-^--^') — t, then, for all A" extending A. A' 
overl{Sj), ^5^, |;(5'^.)(yl") is well-defined. 

(6) For all i G [1, k] such that Si C Ve, 'Ps^\i{S,) = 'Ps,\i{S,)nVE \\Vn- 

(7) For all i G [1, k] such that Si C Vd, Ts, \ i(s,) = ^s, \ i{S,)nVD \\Ve ■ 

The technical resuhs of Proposition 8 ensure that, in the fohowing semantic answer to 
a query (see Definition 30), 

• all quantities Vs\i{s){-^-^') and J^s\i(S){-^--^') used are defined (thanks to items 3 to 
5 in Proposition 8); 

• all eliminations over restricted domains are defined because the restricted domains 
used are never empty (items 1 and 2 in Proposition 8); 

• the conditional distributions used coincide with a conditioning defined directly from 
the controlled plausibility and feasibility distributions Vy^ \ \ v^, and JFy^ 1 1 (items 6 
and 7 in Proposition 8). This is useful because this guarantees that Vs \i(s)i^-^') 
and J^g^i(^g^{A.A'), which a priori require the notion of completion to be written, are 
actually independent of the notion of completion, which is arbitrarily added to the 
basic information expressed in a PFU network. We use Vs\i(s) and Ts\i{S) instead 
of conditional controlled distributions Vs \ i{s)nVE \\Vd and | i(^s)r\VD \\ Ve notation 
convenience and to explicitly represent that Vs \ i{s)nVE || Vb and \ i{s)nVD \ \Ve ^'^ 
depend on the assignment of Vd — liS) and Ve — 1{S) respectively. 
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Definition 30. The semantic answer Sem-Ans{Q) to a query Q 
of the set Vfr of free variables of Q defined by^^ 



{M, Sov) is a function 



Sem-Ans{Q){A) 



tfJ^VfM) = f 
Qs{N', Sov, A) otherwise, 



with Qs inductively defined by: 



(1) 
(2) 



Qs{Af,9,A) =Uv{A) 
Qs{N, {op, S) . Sov, A) = 

QsiAf,Sov,A.A') 



mm 

A' e dom(S) 
^s\ns)iA.A') = 

max 

A' e domiS) 
^s\i(sM.A') = t 

A' G dom(S) 
Vsii(sM.A')j^Op 



Qs{M, Sov,A.A') 



if (S C Vd) a {op = mill), 
if {S C Vd) a {op = max), 



{Vs\KsM-A') (S)puQs{M,Sov,A.A')) if {S c Ve). 



In other words, each step involving decision variables (first two cases) corresponds to an 
optimization step among the feasible choices, and each step involving environment variables 
(third case) corresponds to a lottery (von Neumann & Morgenstern, 1944) such that the 
rewards are the Qs {M, Sov, A. A'), and such that the plausibility attributed to a reward is 
Vs I i[s) {-A-A') (the formula looking like (Bui {Pi "^pu Ui) is the expected utility of this lottery). 
When a set of decision variables S is eliminated, a decision rule for S can be recorded, using 
an argmax (resp. an argmin) if max (resp. min) is performed. 

Example. What is the maximum investment Peter can expect, and which associated deci- 
sion(s) should he make if he chooses the menu without knowing who will attend? To answer 
this question, we can use a query in which bpj, bpM, epj, and epM are eliminated before 
mc and w to represent the fact that their values are not known when the menu is chosen. 
This query is: 

{M, (max, {mc, w}).{®u, {bpj, bpM, epj, epu}))- 

The answer is $6K, with {mc, meat). {w, red) as a decision. If Peter knows who comes, the 
query becomes 

(A/", (©„, {bpj, bpM}).{max, {mc, tt;}).(e„, {epj, ep^}))- 

and optimal values for mc and w can depend on bpj and bpM- The answer is $2()K with 
a $20K gain from the observability of who is present. The decision rule for {mc, w} is 
{mc, meat). {w, red) if John is present and Mary is not, {mc, fish). {w, white) otherwise. 
Consider the query introduced at the beginning of Section 6.1: 

{M, (min, {mc}).(©„, {bpj, 6pM}).(max, {w}).{®u, {ep,j, epi/ })). 

The answer is —oo: in the worst main course case, even if Peter chooses the wine, the sit- 
uation can be unacceptable. In order to compute the expected utility for each menu choice, 
we can use a query in which mc and w are free variables: 



10. is the unfeasible value, cf. Definition 6. 
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(A/", {®u,{bpj, bpM , epj , epM})) ■ 

The answer is a function of{mc,w}. These examples show how queries can capture various 
situations in terms of partial observabilities, optimistic/pessimistic attitude, and parameters 
in the decision process. 

6.3 Operational Answer to a Query 

The quantities Vs \ i(s) (^-^O \ i{S) (^-^O involved in the definition of the semantic an- 

swer to a query are not directly available from the local functions and can be very expensive 
to compute. For instance, with probabilities, Vs \ i(S){^-^') equals Vs^i(s){^-^')/'Pi(S){^)- 
Computing rs,i{s){A-^') = Eyi"edom(y-(5u/{S))) typically requires time 

exponential in |y — (S" U ^(>S'))|. Moreover, such quantities must be computed at each node 
of the decision tree. Fortunately, there exists an alternative definition of the answer to 
a query, which can be directly expressed using a PFU instance, i.e. the expressed local 
plausibility, feasibility, and utility functions. 

Definition 31. The operational answer Op-Ans{Q) to a query Q = (TV, Sov) is a function 
of the free variables ofQ: if A is an assignment of the free variables, then {Op-Ans{Q)){A) 
is defined inductively as follows: 

{Op-Ans{Q))iA) = Qo (A/", Sov, A) 

Qo{M, {op, S) . Sov, A) = op^,gdom(5) Qo (A/", Sov, A.A') (9) 



QoiM,9,A) =[{ A Fi]*[ Pi](E)pu{ (E)u UA] (A). (10) 




By Equation 10, if all the problem variables are assigned, the answer to the query is the 
combination of the plausibility degree, the feasibility degree, and the utility degree of the 
corresponding complete assignment. By Equation 9, if the variables are not all assigned and 
{op, S) is the leftmost operator-variable(s) pair in Sov, the answer to the query is obtained 
by eliminating S using op as an elimination operator. Again, optimal decision rules for the 
decision variables can be recorded if needed, using argmin and argmax. Equivalently, by 
considering a sequence of operator-variable(s) pairs as a sequence of variable eliminations, 
Op-Ans{Q) can be written: 



Op-Ans{Q) = Sov y^^A^Fij ★ ®p Pi j (g)p„ ^^^^^i J j ■ 
It shows that Op-Ans{Q) actually corresponds to the generic form of Equation 8. 
6.4 Equivalence Theorem 

Theorem 2 proves that the semantic definition Sem-Ans{Q) gives semantic foundations to 
what is computed with the operational definition Op-Ans{Q). 

Theorem 2. If the plausibility structure is conditionable, then, for all queries Q on a PFU 
network, Sem-Ans{Q) = Op-Ans{Q) and the optimal policies for the decisions are the same 
with Sem-Ans{Q) and Op-Ans{Q). 
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In other words, Theorem 2 shows that it is possible to perform computations in a 
completely generic algebraic framework, while providing the result of the computations with 
decision-theoretic foundations. Due to this equivalence theorem, Op-Ans{Q) is denoted 
simply by Ans{Q) in the following. Note that the operational definition applies even in 
a non-conditionable plausibility structure. Giving a decision-theoretic-based semantics to 
Op-Ans when the plausibility structure is not conditionable is an open issue. 

6.5 Bounded Queries 

It may be interesting to relax the problem of computing the exact answer to a query. 
Assume that the leftmost operator- variable(s) pair in the sequence Sov is (max, {x}), with 
X a decision variable. From the decision maker point of view, computing decision rules 
providing an expected utility greater than a given threshold 6 may be sufhcient. This is 
the case with the E-MAJSAT problem, defined as Given a boolean formula over a set 
of variables V = Vd U Ve, does there exist an assignment of Vo such that the formula is 
satisfied for at least half of the assignments of Ve ?" Extending the generic PFU framework 
to answer such queries is done in Definitions 32 and 33, which introduce bounded queries. 

Definition 32. A bounded query B-Q is a triple {Af, Sov,6), such that {J\f,Sov) is a query 
and 6 € (9 is the threshold). 

Definition 33. The answer Ans{B-Q) to a bounded query B-Q = {J\f, Sov,6) is a boolean 
function of the free variables of the "unbounded" query Q = (M, Sov). For every assignment 
A of these free variables, 



As the threshold 6 may be used to prune the search space during the resolution, com- 
puting the answer to a bounded query is easier than computing the answer to an unbounded 
one. 

6.6 Back to Existing Frameworks 

Let us consider again the frameworks of Section 3. Solving a CSP (Equation 1) or a totally 
ordered soft CSP corresponds to the query Q = (M, (max, V)), with M the PFU network 
corresponding to the CSP and V the set of variables of the CSP. Computing the probability 
distribution of a variable y for a Bayesian network (Equation 2) modeled as M corresponds 
to Q = {Af, {+,V — {y}). These examples are mono-operator queries, involving only one 
type of elimination operator. 

Consider multi- operator queries. The search for an optimal policy for the stochastic 
CSP associated with Equation 4 is captured by a query such as Q = (AA, (max, {di, ^2}) 
.(-|-, {si}).(max, {(^3, (i4}).(-|-, {52})). The query on influence diagrams of Equation 5 and 
the query on valuation networks of Equation 6 are captured the same way. 

For a finite horizon MDP with T time-steps (Equation 7), the query looks like Q = 
(TV, (max, {di}).{®u, {s2})-(max, {^2}) . . . (©«, {sT}).(max, {dr})), where = + with prob- 
abilistic MDPs and 0^ = min with pessimistic possibilistic MDPs. The initial state si is a 
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free variable. With a quantified CSP or a quantified boolean formula, elimination operators 
min and max are used to represent V and 3. 
More formally, we can show: 

Theorem 3. Queries and bounded queries can be used to express and solve the following 
list of problems: 

1. SAT framework: SAT, MAJSAT, E-MAJSAT, quantified boolean formula, stochastic 
SAT (SSAT) and extended-SSAT (Littman et al, 2001). 

2. CSP ( or CN) framework: 

• Check consistency for a CSP (Mackworth, 1977); find a solution to a CSP; count 
the number of solutions of a CSP. 

• Find a solution of a valued CSP (Bistarelli et al., 1999). 

• Solve a quantified CSP (Bordeaux & Monfroy, 2002). 

• Find a conditional decision or an unconditional decision for a mixed CSP or a 
probabilistic mixed CSP (Fargier et al., 1996). 

• Find an optimal policy for a stochastic CSP or a policy with a value greater than 
a threshold; solve a stochastic COP (Constraint Optimization Problem) (Walsh, 
2002). 

3. Integer Linear Programming (Schrijver, 1998) with finite domain variables. 

4- Search for a solution plan with a length < k in a classical planning problem (STRIPS 
planning, Fikes & Nilsson, 1971; Challab et al, 2004). 

5. Answer classical queries on Bayesian networks (Pearl, 1988), Markov random fields 
(Chellappa & Jain, 1993), and chain graphs (Frydenberg, 1990), with plausibilities 
expressed as probabilities, possibilities, or K-rankings: 

• Compute plausibility distributions. 

• MAP (Maximum A Posteriori hypothesis) and MPE (Most Probable Explana- 
tion). 

• Compute the plausibility of an evidence. 

• CPE task for hybrid networks (Dechter & Larkin, 2001 ) ( CPE means CNF Prob- 
ability Evaluation, a CNF being a formula in Conjunctive Normal Form). 

6. Solve an influence diagram (Howard & Matheson, 1984). 

7. With a finite horizon, solve a probabilistic MDP, a possibilistic MDP, a MDP based on 
K-rankings, completely or partially observable (POMDP), factored or not (Puterman, 
1994; Monahan, 1982; Sabbadin, 1999; Boutilier et al., 1999, 2000). 
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6.7 Towards More Complex Queries 

Queries can be made more complex by relaxing some assumptions: 

• In the definition of queries, the order ^„ on is assumed to be total. Extending 
the results to a partial order is possible if {Eu, :<u) defines a lattice (partially ordered 
set closed under least upper and greatest lower bounds) and if (g)p„ distributes over 
the least upper bound lub and greatest lower bound gib (i.e. p ®pu lub{ui,U2) = 
lub{p ®pu ui,p i^pu U2) and p ®pu glb{ui,U2) = glb{p (S)pu ui,p ®pu U2)). This allows 
semiring CSPs (Bistarelli et al., 1999) to be captured in the framework. We believe 
that other extensions to partial orders on utilities should allow algebraic MDPs (Perny 
et al., 2005) to be captured. 

• We can try to relax the no-forgetting assumption, as in limited memory influence 
diagrams (LIMIDs, Lauritzen & Nilsson, 2001), which show that this can be relevant 
for decision processes involving multiple decision makers or memory constraints on 
the policy recording. In such cases, optimal decisions can become nondeterministic 
(decisions such as "choose x = with probability p and x = 1 with probability 1—p"). 

• The order in which decisions are made and environment variables are observed is 
total and completely determined by the query. One may wish to compute not only 
an optimal policy for the decisions, but also an optimal order in which to perform 
decisions, without exactly knowing the steps at which other agents make decisions or 
the steps at which observations are made. Work on influence diagrams with unordered 
decisions (Jensen &: Vomlelova, 2002) is good starting point to try and extend our 
work in this direction. 

While it should be possible to relax the assumption that variables have a finite domain, 
doing this is nontrivial, since transforming (Bu = + into integrals is not straightforward, 
and performing min- or max-eliminations over continuous domains requires the guarantee 
of existence of a supremum. 

6.8 Summary 

In Section 6, the last element of the PFU framework, a class of queries on PFU networks, has 
been introduced. A decision-tree based definition of the answer to a query has been provided. 
The first main result of the section is Theorem 2, which gives theoretical foundations to 
another equivalent operational definition, reducing the answer to a query to a sequence of 
eliminations on a combination of scoped functions. The latter is best adapted to future 
algorithms, because it directly handles the local functions defined by a PFU network. The 
second important result is Theorem 3, which shows that many standard queries are PFU 
queries. Overall, the PFU framework is captured by Definitions 14, 16, and 17 for algebraic 
structures. Definition 26 for PFU networks, and Definitions 27 and 31 for queries. 

7. Gains and Costs 

A better understanding Theorem 3 shows that many existing frameworks are instances 
of the PFU framework. Through this unification, similarities and differences between ex- 
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isting formalisms can be analyzed. For instance, by comparing VCSPs and the optimistic 
version of finite horizon possibilistic MDPs through the operational definition of the answer 
to a query, it appears that a finite horizon optimistic possibilistic MDP (partially observable 
or not) is a fuzzy CSP: both can indeed be represented as a query Q whose operational 
answer looks like maxy(min(pg$ (^), where y is a set of variables and <^ is a set of scoped 
functions. Techniques available for solving fuzzy CSPs can then be used to solve finite 
horizon optimistic possibilistic MDPs. 

Prom the complexity theory point of view, studying the time and space complexity for 
answering queries of the form of Equation 8 can lead to upper bounds on the complexity 
for several frameworks simultaneously. One may also try to characterize which properties 
lead to a given theoretical complexity. 

Increased expressive power The expressive power of PFU networks is the result of 
a number of features: (1) flexibility of the plausibility /utility model; (2) flexibility of the 
possible networks; (3) flexibility of the queries in terms of situation modeling. This enables 
queries on PFU networks to cover generic finite horizon sequential decision problems with 
plausibilities, feasibilities, and utilities, cooperative or adversarial decision makers, par- 
tial observabilities, and possible parameters in the decision process modeled through free 
variables. 

As none of the frameworks indicated in Theorem 3 presents such flexibility, for every 
subsumed formalism X indicated in Theorem 3, it is possible to find a problem which can 
be represented with PFUs but not directly with X. More specifically, compared to influence 
diagrams (Howard & Matheson, 1984; Jensen & Vomlelova, 2002; Smith et al., 1993; Nielsen 
& Jensen, 2003; Jensen et al., 2004) or valuation networks (VNs, Shenoy, 1992, 2000; 
Demirer &: Shenoy, 2001), PFUs can deal with more than the probabilistic expected additive 
utility and allow us to perform eliminations with min to model the presence of adversarial 
agents. Thus, quantified boolean formulas cannot be represented with influence diagrams 
or VNs, but are covered by PFU networks (see Theorem 3). Moreover, PFU networks use a 
DAG which captures the normalization conditions of plausibilities or feasibilities, whereas 
with VNs, this information is lost. Compared to sequential influence diagrams (Jensen 
et al., 2004) or sequential VNs (Demirer &; Shenoy, 2001), PFUs can express some so-called 
asymmetric decision problems (problems in which some variables may not even need to be 
considered in a decision process) by adding dummy values to variables. 

Actually, some simple problems which can be expressed with PFUs cannot be apparently 
directly expressed in other frameworks. The simple instance "feasibilities with normalization 
conditions + hard requirements" is not captured by any of the subsumed frameworks. For 
example, using a CSP to model it would result in a loss of the information provided by 
the normalization conditions on feasibilities. The same occurs for influence diagrams - 
like sequential decision processes based on possibilistic expected utility, which could be 
called possibilistic influence diagrams. Similarly for stochastic CSPs without contingency 
assumption. 

The cost of greater flexibility and increased expressive power is that the PFU framework 
cannot be described as simply and straightforwardly as, for example, constraint networks. 

Generic algorithms Section 8 shows that generic algorithms can be built to answer 
queries on PFU networks. As previously said, building generic algorithms should facilitate 
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cross-fertilization in the sense that any of the subsumed formahsms will directly benefit 
from the techniques developed on another subsumed formalism. This fits into a growing 
effort to generalize resolution methods used for different AI problems. For example, soft 
constraint propagation drastically improves the resolution of valued CSPs; integrating such 
a tool in a generic algorithm on PFUs could improve the resolution of influence diagrams. 
Using abstract operators may enable us to identify algorithmically interesting properties, 
or to infer necessary or sufficient conditions for a particular algorithm to be usable. 

However, one could argue that some techniques are highly specific to one formalism or 
to one type of problem, and that, in this case, dedicated approaches certainly outperform 
a generic algorithm. A solution for this can be to characterize the actual properties used 
by a dedicated approach, in order to generalize it as much as possible. Moreover, even if 
specialized schemes usually improve over generic ones, there exist cases in which general 
tools can be more efficient than specialized algorithms, as shown by the use of SAT solvers 
for solving STRIPS planning problems (Sang, Beame, & Kautz, 2005). 

8. Algorithms 

The ability to design generic algorithms is one of the motivations for building the PFU 
framework, and some choices are justified by algorithmic considerations. We present generic 
algorithms that answer arbitrary PFU queries. 

8.1 A Generic Tree Search Algorithm 

The operational definition of the answer to a query Q actually defines a naive exponential 
time algorithm to compute Ans{Q) using a tree-exploration procedure, with a variable 
ordering given by Sov, that collects elementary plausibilities, feasibilities, and utilities. 
More precisely, for each assignment A of the free variables of Q, a tree is explored. Each 
node in this tree corresponds to a partial assignment of the variables. The value of a leaf 
is provided by the combination of the scoped functions of the PFU network, applied to 
the complete assignment defined by the path from the root to the leaf. Depending on 
the operator used, the value of an internal node is computed by performing a min, max, 
or ®u operation on the values of its children. The root node returns {Ans{Q)){A). The 
corresponding pseudo-code is given in Figure 2. For a query {J\f,Sov), the first call is 
TreeSearchAnswerQ(A/', Sov). It returns a function of the free variables. 

If we assume that every operator returns a result in a constant time, then the time 
complexity of the algorithm is 0{m ■ n ■ ln[d) ■ d"), where d stands for the maximum domain 
size, n stands for the number of variables of the PFU network, and m stands for the number 
of scoped functions. 

The space complexity is polynomial (it can be shown to be linear in the entry data 
size). Hence, computing the answer to a bounded query is PSPACE. Moreover, given that 
the satisfiability of a QBF is a PSPACE-complete problem which can be expressed as a 
bounded query (cf. Theorem 3), it follows that computing the answer to a bounded query 
is PSPACE-hard. Being PSPACE and PSPACE-hard, the decision problem that consists of 

11. The factor n ■ ln{d) corresponds to an upper bound on the time needed to get fiA) for a scoped function 
ip represented as a table (of size < d"). 
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TreeSearchAnswerQ((y, G, P, F, [/), Sov) 
begin 

foreach A e dom{Vfr) do ^ AnswcrQ((F, G, P, F, U),Sov, A) 
return 
end 

AnswerQ((y, G, P, P, U),Sov, A) 
begin 

if 5*011 = then return ((Af.g-F Pi) * (®pp,gp ^i) ®pm (®«c/,g;7 
else 

(op, S).Sov' <— Sow 
choose X E S 

if S* = {x} then Sow 5oi;' else Sov ^ (op, S — {x}).Sov' 
dom ^ dom{x) 
res ^ 

while dom ^ do 

choose a £ dom 
dom <— dom — {a} 

res ^ op {res, AnswerQ((y, G, P, P, U), Sov, A.{x, a))) 
L return res 

end 

Figure 2: A generic tree search algorithm for answering a query Q = 
{iV,G,P,F,U),Sov) 

answering a bounded query is P SPACE- complete. This result is not surprising, but it gives 
an idea of the level of expressiveness which can be reached by the PFU framework. More 
work is needed to identify subclasses of queries with a lower complexity, although many are 
already known. 

8.2 A Generic Variable Elimination Algorithm 

Quite naturally, a generic variable elimination algorithm (Bertele & Brioschi, 1972; Shenoy, 
1991; Dechter, 1999; Kolhas, 2003) can be defined to answer queries on a PFU network. 

8.2.1 A First Naive Scheme 

This first naive variable elimination algorithm is given in Figure 3. It eliminates variables 
from the right to the left of the sequence Sov of the query, whereas with the tree search 
procedure, variables are assigned from the left to the right. This right-to-left processing 
entails that the algorithm naturally returns a function of the free variables of the query. 
The first call is VarElimAnswerQ((U, G, P, F, U), Sov). 

The version presented in Figure 3 is actually a very naive variable elimination scheme 
with time and space complexities 0{m ■ n ■ ln{d) ■ d") and 0{m ■ d") respectively: it begins 
by combining all the scoped functions before eliminating variables, whereas the interest of 
a variable elimination algorithm is primarily to use the factorization into local functions. 
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VarElimAnswerQ((y, G, P, F, U),Sov) 
begin 

^ ((A_F,eF Fi) ★ (®pp_gp Pi) ®pM (®«;7,e;7 C^j)) 
while Sov ^ do 

Sov' .{op, S) ^ Sow 
choose X & S 

if S* = {x} then 5ow ^ So-y' else Sow ^ Sov'. {op, S — {x}) 
|_ (po ^ op,, 
return ipo 
end 

Figure 3: A first generic variable elimination algorithm for answering a query Q = 
{{V,G,P,F,U),Sov) 



8.2.2 Improving the Basic Scheme 

The algorithm of Figure 3 works on a unique global function defined by the combination 
of all the plausibility, feasibility, and utility functions (first line), whereas a factorization is 
available. To improve this scheme, the properties of the algebraic structure can be used. 
In the sequel, we denote by (resp. $~^) a scoped function which has (resp. does not 
have) X in its scope. Moreover, we extend every combination operator Cg) on U {{>} by 
setting () ^ e = eC^O = (combining anything with something unfeasible is unfeasible 

t00).12 

First, in order to use the factorization of plausibilities and feasibilities, we can use the 
properties below, which come from the right monotonicity of ®pu^ the distributivity of 
over ©u, and the definition of the truncation operator 

min^ {P-^ ®pu U) = P-"" ®pu (miua; U) 

maxa, {P-"" ®pu U) = P""" ®pu (max^ U) 

^ ®UX {P ^ ®pU U) = P ^ (S>pu (©MX 

miux {F~^ -kU) = F~^ ★ (min^; ^7) 
maxa; {F~^ -kU) = F~^ -k (max^; U) 
( ®uAF-''*U) = F-^*{(Bu.U). 
They express that when a variable x is eliminated, it is not necessary to consider plausibility 
functions or feasibility functions that do not have x in their scope. 

However, it is necessary to add some axioms on the expected utility structure, since 
in the general case, an expression such as ®ux (-P"*"^ ®pu {U~^ ®u U^^)) cannot be decom- 
posed. We give two axioms, Axl and Ax2, each of which is a sufficient additional condition 
to exploit the factorization of utility functions. 

(Axl) {Ep, ^p) = {Eu, ^u), ©p = ©«, 'S)p = (S)u = 'S)pu 
(Ax2) e„ = (g)u on Eu (but not on £'„ U {<>})• 



12. An operator op can be used both as a combination operator between scoped functions and as an elimina- 
tion operator over some variables. In this case, the extension of op used as a combination operator creates 
an operator op' such that op'(e,0) = 0, whereas the extension of op used as an elimination operator 
creates an operator op" such that op" {e, 0) = e. op' and op" coincide on E but differ on i? U {(>}• 
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Among the cases in Table 1, rows 2, 3, 5, 6 satisfy Axl, whereas rows 1, 4, 7, 8 satisfy 

Ax2. Axl and Ax2 enable us to write: 

min^ * ([/"^ (8)„ = C/"^ ®„ (min^ * C/+^) 

max^ * (C/-^ ^7+^)) = C/"^' (g)„ (max^ * . 



and 



(8)„, -P+^ (8)p,. f/+^') with Axl 



®f« ^ - \ ((e^^ p+x) p+- with Ax2. 

Hence, when eliminating a variable x, it is not necessary to consider utility functions which 
do not have x in their scope. 

We present an algorithm when Axl is satisfied. When Ax2 holds, working on plau- 
sibility/utility pairs {p,u) allows Axl to be recovered: this is used, for example, to solve 
influence diagrams (Ndilikilikesha, 1994). When Axl is satisfied, there is actually only one 
set E = Ep = Eu, one order ^=^p=^n) one combination operator ® = = <8>m = ®pu, 
and one elimination operator © = ©p = ©„. Rather than express feasibilities on {t,f}, we 
can express them on {1^;, 0} by mapping t onto 1e and / onto 0: this preserves the value 
of the answer to a query, since f * u = () u and t-ku = 1e 

The improved variable elimination algorithm is shown in Figure 4. To answer a query 
Q = [[V, G, P, F, U),Sov), the first call is Axl-VarElimAnswerQ(P UFUU, Sov). It 
returns a set of scoped functions whose ^-combination equals Ans{Q). This time, the 
factorization available in a PFU network is exploited, since when eliminating a variable x, 
only scoped functions involving x are considered. 



Axl-VarElimAnswerQ($, Sov) 
begin 

if Sov = ^ then return <i> 
else 

Sov' .{op, S) ^ Sov 
choose X & S 

if S = {x} then Sov ^ Sov' else Sov <— Sov' .{op, S — {x}) 

^ {(^ e $ I a; e sc{ip)} 
fo ^ op^ («)^g$+-¥') 
$ ^ ($ _ $+^) U {(^o} 
return Axl-VarElimAnswerQ{^, Sov) 

end 



Figure 4: Variable elimination algorithm when Axl holds (<^: set of scoped functions) 

When Axl holds, the algorithm is actually a standard variable elimination algorithm on a 
commutative semiring. As for classical variable elimination algorithms, the time complexity 
of this algorithm is in 0{m ■ n ■ln{d) ■ d'""''^), where w is the tree- width (Bodlaender, 1997; 
Dechter & Fattah, 2001) of the network of scoped functions, constrained by the elimination 
order imposed by Sov. Yet, its space complexity is also exponential in this tree-width. 

8.3 Other Approaches 

Starting from the generic tree-search algorithm of Section 8.1, bound computations and local 
consistencies (Mackworth, 1977; Cooper &: Schiex, 2004; Larrosa &: Schiex., 2003) can be 
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integrated in order to prune the search space. Local consistencies can improve the quahty of 
the bounds thanks to the use of smaller local functions. Techniques coming from quantified 
boolean formulas or from game algorithms (such as the a/J-algorithm) can be considered to 
more efficiently manage bounds when min and max operators alternate. Caching strategies 
exploiting the problem structure (Darwiche, 2001; Jegou & Terrioux, 2003) are also obvious 
candidates to improve the basic tree search scheme. Additional axioms Axl and Ax2 can 
be useful in this direction. Heuristics for the choice of the variable to assign when a pair 
{op, S) is encountered, as well as heuristics for the value choices, may also speed up the 
search. 

In another direction, approximate algorithms using sampling and local search could also 
be considered: sampling when eliminations with + (+, and not (Bu) are performed, local 
search when eliminations with min or max are performed. 

9. Conclusion 

In the last decades, AI has witnessed the design and study of numerous formalisms for 
reasoning about decision problems. In this article, we have built a generic framework 
to model sequential decision making with plausibilities, feasibilities, and utilities. This 
framework covers many existing approaches, including hard, valued, quantified, mixed, and 
stochastic CSPs, Bayesian networks, finite horizon probabilistic or possibilistic MDPs, or 
influence diagrams. The result is an algebraic framework built upon decision-theoretic 
foundations: the PFU framework. The two facets of the PFU framework are explicit in 
Theorem 2, which states that the operational definition of the answer to a query is equivalent 
to the decision tree-based semantics. This is the result of a design that accounts both for 
expressiveness and for computational aspects. 

Compared to related works (Shenoy, 1991; Dechter, 1999; Kolhas, 2003), the PFU frame- 
work is the only framework which directly deals with different types of variables (decision 
and environment variables), different types of local functions (plausibilities, feasibilities, 
utilities), and different types of combination and elimination operators. 

Prom an algorithmic point of view, generic algorithms based on tree search and variable 
elimination have been described. They prove that the PFU framework is not just an ab- 
straction. The next step is to explore ways of improving these algorithms, so as to generalize 
techniques that are used in formalisms subsumed by the PFU framework. Along this line, 
a generic approach to query optimization has lead to the definition of original architectures 
for answering queries, called niulti- operator cluster trees and multi- operator cluster DAGs. 
These can be applied to QBFs and other structures compatible with Axl (Pralet, Schiex, & 
Verfaillie, 2006a), as well as influence diagrams and other structures satisfying Ax2 (Pralet, 
Schiex, & Verfaillie, 2006b). 
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Appendix A. Notations 

See Table 2. 

Symbol Meaning 



© 


Elimination operator 


jj 


Elimination operator on plausibilities 




Elimination operator on utilities 


® 


Combination operator 




Combination operator for plausibilities 




Combination operator for utilities 




Combination operator between plausibilities and utilities 


— I' 


Partial order on plausibilities 


<u 


Partial order on utilities 


•k 


Truncation operator 





Unfeasible value 


Ve 


Environment variables 


Vd 


Decision variables 


dom{x) 


Domain of values of a variable x 


dom{S) 




G 


Directed Acyclic Graph (DAG) 


pacix) 


Parents of x in the DAG G 


ndcix) 


Non-descendant of x in the DAG G 


Cb(G) 


Set of environment components of G 


Cd{G) 


Set of decision components of G 


Pi 


Plausibility function 


Fi 


Feasibility function 


Ui 


Utility function 


Fact{c) 


Pi or Fi factors associated with a component c 


sc{lp) 


Scope of a local function Lp 


Vs 


Plausibility distribution over S 


'Ps, 1 s. 


Conditional plausibility distribution of 5*1 given 5*2 




Feasibility distribution over 5* 




Conditional feasibility distribution of Si given ^2 


Sov 


Sequence of operator-variable(s) pairs 


Sem-Ans((3) 


Semantic answer to a query Q (decision trees) 


Op-Ans(g) 


Operational answer to a query Q 


Ans(Q) 


Answer to a query Q 



Table 2: Notation. 
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Appendix B. Proofs 

Proposition 1 A plausibility distribution Vs can be extended to give a plausibility distri- 
bution Vs' over every S' C S, defined by Vs' = ®pg_g, Vs- 

Proof. Given that (Bp is associative and commutative, (Bpg, Vs' — ©pg/ {®ps_si 'Ps) — ®ps ~ Ip- 
Thus, Vs' ■ do'm{S') Ep is a plausibihty distribution over S' . □ 



Proposition 2 The structures presented in Table 1 are expected utility structures. 
Proof. It is sufficient to verify each of the required axioms successively. □ 

Proposition 3 If {Ep,(Bp,'S>p) is a conditionable plausibility structure, then all plausibil- 
ity distributions are conditionable: it suffices to define Vsi | S2 by Vs^ \ $2 (^) ~ max{p G 
Ep I Vs^,S2{^) =P®p 7^52 (^)} for all A G dom{Si U ^2) satisfying Vs^i^) + %■ 

Proof. Let Vs be a plausibihty distribution over S. For all S2 disjoint subsets of S and for all 
A e dom{Si U 52) satisfying 'Ps2{A) ^ Op, let us define Vs,\sA^ = max{p e £'p | Psi.Ss (^) = 
V ®p Vsi {A)]. We must show that the Vs, \ S2 functions satisfy axioms a, b, c, d, e of Definition 18. 

(a) By definition of Vs, \ S2 by distributivity of (E>p over ©p, we can write 

Vs2 = ©psi Ps,,S2 = ©P5i (Ps, I S2 ®P T'sJ = (©PS, Vs, I sj ®P PS2- 
As ■Psj :<p ■Psj, we can infer that ©p^, Vsi \ S2 ^p Ip- Let A2 be an assignment of S2 satisfying 
■^52(^2) 7^ Op. Assume that the hypothesis (H): "®pg, Vsi 182(^2) ^p Ip" holds. 

Then, for aU A, G dom(5i), Psi,52 (^1-^2) <p ^52(^2), since if 7'si,S2 (^1-^2) = ^52(^2), 
then Vg, I 52(^1-^2) = Ip, which implies that ©p^^ "Psi | 52(^2) hp Ip by the monotonicity of 
©p. Moreover, (H) imphes that there exists a unique p ^ Ep satisfying (©p^^ Vs, \ 52(^2)) ©p 
p = Ip. Combining this equation by 7^52(^2) gives 7^52(^2) ©p ^^52 (^2) ®pP ^ "^32 (^2), i.e. 
"^52(^2) ®p (Ip ®pP) = 'Ps2(^2). This implies that Ip ©pp ^p Ip. Given that Ip ®pp >p Ip 
(by monotonicity of ©p), we obtain Ip ©p p = Ip. We analyze two cases. 

• If p ^p Ip, there exists a unique p' satisfying p'©pp = Ip. As both (©p^^ | 52(^2)) ©pP 
= Ip and Ip ©pp = Ip, this entails that ©pg, Vs, \ 32(^2) — Ip, which contradicts (H). 

• If p = Ip, then Ip ©p Ip ~ Ip. This entails that ©p is idempotent. Let dam' be a subset 
of dom{Si) such that ©p^^^^^^^, 7'5i,S2 (^1-^2) = 7^52(^2). Let A'^ e dam'. We can 
write: 

7'Si.S2(^'l-^2) ©p {®PA,edom'-{A',] VsuS2{MA2)) = 7^52(^2) 

7^51,52 (^i -^2) ©p (©PAiedom' ^51,52(^1-^2)) = ■Ps2(^2) (as ©p is idempotent) 

As 7^51.52 (^'1-^2) ^p 7^52(^2), there exists a unique p" S i?p such that 7'5i,s2(^i-^2)ffip 
p" - 7's2(^2). Therefore, ©p^^^.^^, 7^51,52 (^1-^2) - ®PA,edom'-{A',}'Psls2{A^.A2), 

which gives ®pA,edom'-{A',] Ps,,S2{Ai.A2) = 7^52 (^2). 

The assumption ©p^^^^^^^, ^'5^^52(^1.742) = 7^52(^2) holds for dam' = dom{Si). Re- 
cursively applying the previous mechanism by removing one assignment in dam' at each 
iteration leads to ©p^jg^om' ^S'i,S2(^i-^2) = 7^52(^2) with \dom'\ = 1, i.e. it leads to 
7^51,52 (A'/. ^2) = 7^52(^2) with dam' = {A'l}. As a result, we obtain a contradiction. 

In both contradiction with (H) is obtained, so ®pg, Vs, \ 52(^2) = Ip- 

(b) 7'5i =Vs,\^ ©p 7^0 = 7^51 1 ©p (©P5 Vs) =Vs,\$ ®p lp^Vs,\$. 
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(d) Let A G dom{Si U ^2 U ^3) satisfying Vs^.,s, {A) ^ %. Then, Vs.^s, \ s, {A) = Vs, \ 5.,S3 (^) ®p 
7^52 1 S3 (^) holds, because: 

• If Vsi,S2,S3{A) -^p Vs^iA), then, there exists a unique p € Ep such that 'Psi,S2,S3{A) = 
p^pVsM). As both Vs,,S2,sAA) = VsuS2 \ sAA)®pVsM) (by definition oiv'suS^ I S3) 
and 'PsuS2,S3{A) = Vs^ \ S2,S3(^)®p^S2 | S3 (^) ®p ^S3 (^) (by definition of T's, | 32,83 and 
^S2 1 S3)' this imphes that 7^51,52 1 S3(^) = ^Si | S2,S3(^) ®p ^Sa | S3(^)- 

• Otherwise, 'Ps-i_,S2,S3{A) = Vs^iA). This imphes that Ip <p 'Psi,S2 1 S3(^) and, as 
■Psi,S2 |S3(^) Ip, that 7'si,S2 |S3(^) = Ip- Similarly, this entails that 7^52 1 S3 (^) = Ip 
and I S2.S3(^) = Ip (the monotonicity of ffip implies that 7^Si,S2,S3 (^) = ^S2,S3(^) = 
Vs^iA)). As Ip = Ip ®p Ip, we get Psi,S2 1 S3(^) = ^Ps^ \ S2,S3(^j ®p ^S2 | S3(^)- 

(c) ®pg^ Vs,,s2 1 S3 = ®PSi (^Si I S2,S3 ^p ^S2 1 S3) (using (d)) 

= (®p5j I S2,S3) ^p 7^S2 1 S3 (because ®p distributes over ©p) 
= ^S2 1 S3 (using (a)) 

(e) Assume that 7'si,S2,S3 = 'Psi \ S3 <8)p '^S2 1 S3 ®p ''^S3- Let A g dom{Si U 6*2 U 53) such that 
VssiA) ^ Op. Then,'7'si.S2 1 S3(^) = 'Ps^\ s^iA) ®p Vs2 1 S3(^) holds, because: 

• If ''^Si.S2,S3(^) 'P.SsiA), there exists a unique p e Ep such that Psi.s2,S3 (^) = 
p ®p U) , and therefore rs^,S2 1 S3 (A) = Vg, \ S3 [A) ®p Vs2 \ S3 (^) ■ 

• Otherwise, we can write Vs^ \ S3 (A) = 7^52 1 S3 {A) — Psi,S2 1 S3 (A) — Ip by using reason- 
ing similar to that of (d), and therefore 7'si,S2 1 S3(^) = ^Si | S3(^) ®p ^S2 | S3(^)- 

□ 

Proposition 4 /(., . | .) satisfies the semigraphoid axioms: 

1. symmetry: I{Si, S2 \ S3) /(S'2, Si | ^3), 

2. decomposition: I{Si, S'2 U ^3 | ^4) I{Si, S2 | S4), 

3. weak union: I {Si, 52 U 53 | 5*4) — > I {Si, 52 | 53 U 54), 

4. contraction: (/(5i, 52 | 54) A I{Si, 53 | 52 U 54)) ^ /(5i, 52 U 53 | 54). 
Proof. 

1. Symmetry axiom: directly satisfied by commutativity of ®p. 

2. Decomposition axiom: assume that I{Si,S2 U 6*3 | S'4) holds. Then 

^Si,S2 I S4 = ©PS3 '^Si,S2,S3 I Si 

= ©PS3 (^Si I S4 ®P ^S2,S3 1 sj (since I{Si,S2 U ^3 I ^4)) 

I Si ®p (®PS3 ^82, S3 1 S4) (by distributivity of ®p over ©p) 

= "PsiISi <^P^S2|S4■ 
Thus, I{Si,S2\Si) holds. 

3. Weak union axiom: assume that I{Si, S'2 U 6*3 | S'4) holds. The decomposition axiom entails 
that /(S'l, S'3 I S'4) is also satisfied. Then 

^Si,S2,S3,S4 = ^Si,S2,S3 1 S4 ®P^S4 (chain rule) 

= Vs, I S4 ®P ^S2,S3 1 S4 ®P T^Si (since I{Si , ^2 U ^3 I S4)) 

= 'Psi I S4 <»P ^S3 1 S4 <^P ^S4 «>p ^S2 1 S3,S4 (chain rule) 

= ^Si,S3 1 S4 ®P '''S4 ®p ^S2 1 S3,S4 (since /(Si, S3 | S'4)) 

= ^Si I S3,S4 <8p "^52 1 S3,S4 ®P "^33, S4 (chain rule). 

Prom axiom (e) in Definition 18, we can infer that 7'si.S2 1 S3.S4 = ^Si | S3,S4 ®p "^s, | S3.S4' i-^- 

/(S'l, S'2 I S3 US4) holds. 
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4. Contraction axiom: assume that I{Si, S2 \ S4) and /(S*!, S'3 | 6*2 U S4) hold. Then 
^Si,S2,S3 1 S4 = ^Si,S3 1 S2,S4 ®P 1 S4 (chain rule) 

= I S2,S4 «)p 1 S2.S4 ®P 1 S4 (since I{Si , ^3 I ^2 U ^4)) 
= ^Si,S2 1 S4 ®P 1 S2,Si (chain rule) 
= ^Si I S4 ®P 1 S4 «)p ^S3 1 S2,S4 (since I{Si,S2 \ 3^)) 
= 'Psi I S4 "^P "^52, S3 1 Si (chain rule). 
Thus, I{Si,S2 U 5*3 I S'4) holds. 

□ 

Theorem 1 (Conditional independence and factorization) Let {Ep, (Bp, 0p) be a condition- 
able plausibility structure and let G be a DAG of components over S. 

(a) If G is compatible with a plausibility distribution Vs over S, then Vs = '^PceC{G) | paa 

(b) If, for all c G C(G), there is a function '~Pc,pac(c) such that '■Pc,pac(c)(,-^ is a plausibility 
distribution over c for all assignments ^ of pac (c), then 75 = <8)p^g^(fj) ^c,pac{c) is a 
plausibility distribution over S with which G is compatible. 

Proof. 

(a) First; if |C(G)| = 1, G contains a unique component ci. Then, ®Pcec(G) ^c|paG(c) = |0 = 
Vci'- the proposition holds for |C(G)| = 1. 

Assume that the proposition holds for all DAGs with n components. Let G be a DAG of 
components compatible with a plausibility distribution Vs and such that |C(G)| — n + 1. 
Let Co be a component labeling a leaf of G. As G is compatible with Vs, we can write 
I{cQ,ndG{co) — paG(co) I paG(co)). As cq is a leaf, ndcico) ~ S — cq, and consequently 
I{co,{S - Co) - _paG(co) baG(co)). This means that Vs-paG(ca)\paG(co) = Vcolpaaico) ®p 
7'(s-co)-pac3(co) Ipqg(co)- Combining each side of the equation by Vpaa{co) gives 

Vs = T'co I pac (co) ®P ^S-co- 

Let G' be the DAG obtained from G by deleting the node labeled with cq. Then for every 
component c € C{G'), paG'{c) = padc) (since the deleted component cq is a leaf). Moreover 
ndc'ic) equals either ndcic) or ndcic) — cq (again, since the deleted component cq is a leaf). In 
the first case {ndo'ic) = ndaic)), the property I{c,ndcic) — pacic) Ipadc)) directly implies 
I{c,ndG'{c)~paG'{c) \po,G'{c))- In the second case (ndcic) = ndG(c) — co), the decomposition 
axiom allows us to write I{c,ndG'ic) ~ paG'{c) \paG'{c)) from I{c,ndG{c) — paG{c) |paG(c)). 
Consequently, G' is a DAG compatible with Vs-co- As |C(G')| = n, the induction hypothesis 
gives Vs-co = ®PceC{G') '^cipaaic), which implies that Vs = <^pc(zc(g) 'Pc\paG(c): as desired. 

(b) Assume that for every component c, ^c,paG{c){A) is a plausibility distribution over c for all 
assignments A of paG{c). For |C(G)| ~ 1, C(G) ~ {ci}. Then, 75 = (Pa is a plausibility 
distribution over ci. Moreover, as 7010 = Ip, we can write 7ciU0|0 = 7ci | ®p70|0i i-C- 
/(ci,0 I 0). Therefore, G is compatible with 7^: the proposition holds for |C(G)| = 1. 

Assume that the proposition holds for all DAGs with n components. Consider a DAG G 
with n + I components. We first show that 7s is a plausibility distribution over S, i.e. 
ffip5 (®Pcec(G) 'Pc,paa{c)) = Ip- Let Co be a leaf component in G. As co is a leaf, the unique 
scoped function whose scope contains a variable in co is 'Pco,paG{co)- distributivity of 

over ©p, this implies that 

®Pco ('^PceCiG) 'fc,paG{c)) = (®Pco '/'co .pac (co) ) '^P {'^Pc(^C{G)-{co} ^c,paG{c))- 

Given that (p^^ j,g^^(^f.^^{A) is a plausibility distribution over cq for all assignments A of paG(co), 
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®VcaVco:P'iG{ca) = Ip- Consequently, 

®Pco (®PcGC(G) ^c,paG{c)) = ®PceC(G)-{co} ^c^paaic)- 

Applying the induction hypothesis to the DAG with n components obtained from G by delet- 
ing Co, we can infer that ®pg_^^ (®Pcec(G)-{co} '^c.paG(c)) = Ip- This allows us to write 

)) = Ip, i.e. ffip5 75 — Ip. 7s is a plausibility distribution 
over S. It remains to prove that G is a DAG of components compatible with 73. Let c G C(G). 
We must show that I{c,ndG(c) — paQ(c) \ paQ(c)) holds. There are two cases: 

1. If c = Co, we must prove that 

lco,ndG(co)—paG(co)\paG(co) ~ 7co|paG(co) ®P 7tic!g (co)— pac (co) | pac (co) ■ 

First; note that 

lco,paG{ca) = ®PS-(coUpaG(co)) (®PcGC(G) ^c,paG(c)) 

= (®PS-(coUpaG(co)) (®PceC(G)-{co} ^c,j}aG{c))) ®p fco,paG{ca) 

(because ®p distributes over ®p and sc{tfco,paG{co)) — "^o UpaG(co) 

= ^®PS-paG{co) ^®Pc(^C(G) ^c,paG{c))) ®p ^ca,paG{co) 

(because ®p distributes over (Bp and ©co ^co,paG(co) = Ip) 

= 7paG(co) ®P '/'co,paG(co). 

From this, it is possible to write: 

lndG(ca)-paG(co)\paG(co) ®p lco\paG(co) ®p IpaG^co) 

— 7ndG(co)-paG(co) I paG(co) ®P 7co,paG (co) 

— lndG{co)-paG{co)\paG(co) ®p lpaG(co) ®P Vco,paG{co) 
= IndG^ca) ®p '^co,paG{co) 

= 7s-{co} ®p <Pco,paG(co) (because co is a leaf in G) 

— {®Pc£C{G)-{co}^c,jmG{c)) ®p 'Pco,paG{co) 
~ ®PceC{G)Vc,paGic) 

= IS- 

Using axiom (e) of Definition 18, this entails that 7ndG(co)-paG(co) I paG(co)'^p7co | pacCc,,) = 
ls-paG{co)\paG{co), i.c, as S* = coUndoico), that /(co,ndG(co) - paG{co)\paG{co)). 

2. Otherwise, c 7^ co- Let G' be the DAG obtained from G by deleting co. G' contains 
n components: by the induction hypothesis, I{c,ndGi{c) — paG'{c)\paG'{c))- As cq 
is a leaf in G, we have co ^ paG{c), which implies that paG'{c) = paG(c). Thus, 
/(c, ndG'[c) -paG{c) \paG{c)). 

(i) If ndG'{c) = ndG{c), then it is immediate that I{c,ndG{c) — paG{c) \ paG{c)). 

(ii) Otherwise, ndG'{c) ^ ndG{c). As cq is a leaf in G, this is equivalent to say that 
ndGifi) = ndG'{c) U co. This means that c is not an ancestor of cq, and a for- 
tiori c ^ paGico). In the following, the four semigraphoid axioms are used to 
prove the required result. From the decomposition axiom, from I{co,ndG{co) — 
paG{cQ) Ipacico)), and from {cUndG'ic)) C ndG{co) (because ndG{co) = S — 
Cq), it follows that /(cq, (c U ndc (c)) ~ pacico) \paG{co)), or, in other words, as 
c n paG(co) = 0, that /(co,cU {ndG'{c) — paG(co)) | paG(co)). Using the weak 
union axiom leads to /(co,c| {ndG'{c) — paG{co)) UpaG(co)) and, using the sym- 
metry axiom, to /(c, cq | (ndc (c) — paG(co)) U paG(co)). As shown previously, 
I{c,ndG'{c)~paG{c) |paG(c)). Together with /(c, cq | (ndG'(c)-paG(co))UpaG(co)), 
the contraction axiom implies that /(c, (ndG'(c) — POg(c)) U cq |paG(c)). As Cq ^ 
paG^c) and ndG{c) = ndG'{c) U cq, this means that I{c,ndG{c) — paG{c)) \ paG{c))- 
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We have proved that G is compatible with 73. Consequently, the proposition holds if there 
are n + 1 components in G, which ends the proof by induction. 

□ 

Proposition 5 Let {Ep^®p,®p) be a conditionable plausibility structure. Then, for all 
n e N*, there exists a unique such that ®p^^^i po = Ip- 

Proof. Let n £ N*- If ffipjgjj^ „] Ip = Ip, then po ~ Ip satisfies the required property- Moreover, in 
this case, the distributivity of ®p over (Bp implies that for all p S £^p, ©pjg[]^ ^^P = P- Therefore, if 
®Pie[i ri] P ~ -'^P' tlicn p = Ip, which shows that po is unique- 
Otherwise, ©pjgjj „] Ip ^ Ip- In this Ip ^p ©p,jg[i „] Ip by monotonicity of ©p, we can 
write Ip -<p ©pjgj]^ „] Ip- The second item of Definition 19 then implies that there exists a unique 
Po e Ep such that Ip = po 8)p (©Pig[i_„] Ip), i-e- such that Ip = ©pig[i.„] Po- □ 

Proposition 6 Let VvEyo be the completion of a controlled plausibility distribution 
II y^. Then, Vve,Vd is a plausibility distribution over U Vd and | y^ = Py^ || y^. 

Proo/. T'yE.Vo = T^Ve \\Vd ®pPo, where po is the element of Ep such that ©Pig[ijrfo„i(y„)|] Po = Ip- 
Then ©py^uvo ^^^i^.vb = ©Pv^uy^ (^^^b II Po) = ©py^ ((®py^ || Vd) ^p Po) = ©py^Po = 

®Pie[i \dom(VD)\] ^ ^P' ^^^^ proves that "Py^.y^ is a plausibility distribution over Ve U Vd- 

As VvEyn = PyB II yo ®p Po and Vve,Vd = "Pve \ Vd ®p 'Pvd, we can write Vve \\Vn ®p Po = 
'Pve I Vo ®P 'Pvn - Moreover, Vvn = ©py^ "PyE.yo = ©py^ (^^-E II Vb ®p Po) = Po- Thus, Vve || y^ ®p 
Po = "Pve I yo ®pPo- Summing this equation |dom(VD)| times with ©p gives 'Py^, || y^ = Pye \ Vo- ^ 

Proposition 7 Let G be a typed DAG of components over Ve U Vb- Let Gp be the partial 
graph of G induced by the arcs of G incident to environment components. Let be the 
partial graph of G induced by the arcs of G incident to decision components. If Gp is 
compatible with the completion of 'Pv^wvd i^^- Definition 22) and Gj is compatible with 
the completion of ^y^ 1 1 y^ , then 

'Pve I Vo = 'Pc I paa (c) and J^-^^^ | y^ = A J^^ | paaic) ■ 

c&Ce{G) ceCoiG) 

Proof. The result is proved only for Vve | Vd (^^"^ proof for Ty^ | y^ is similar). The completion of 
Vve II Vd looks like Vve,Vd = "^Ve II yr> ©pPo- Gp being compatible with this completion. Theorem la 
entails that Pve,Vd = ®Pcgc(g ) I paop (c) ■ ^be decision components are roots in Gp, we can infer, 
by successively eliminating the environment components, that Vvd ~ ®pve ^"^e-Vd = ^PceCoiG ) 

On the other hand, Vvo ~ ®pve i'^yEWVo '^pPo) — Po- This proves that '^pc(zCe,{Gj,)'^'^ ~ 
Po- Therefore, Pve,Vd = 'Pve\Vo ®p Po = i®PceCE{Gp)'^c\paGp{c)) ®pPo- Summing this equation 
|(iom(VD ) I times with ffip gives Pyg I y^ = ®PceCE{Gp)'^c\paGj,{c)- AsC£;(Gp) = C£;(G) and paGp(c) = 
paaic) for every c £ Ce(G), this entails that Py^ | y^ = ®Pc£Ce(G) T^ci paaic)- □ 

Proposition 8 Assume that the plausibility structure used is conditionable. Let Q = 
(Af, Sov) be a query where Sov = {opi, Si) • {op2, S2) • • • (opk, Sk)- Let Vfr denote the set 
of free variables of Q. 
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(1) If Si C Ve and | i{Si){^) is well-defined, then there exists at least one A' G dom{Si) 
satisfying Vs,\i{s,){^-^') + Op. 

(2) If Si C Vd and Tg- 1 /(^.^(A) is well-defined, then there exists at least one A! G dom{Si) 
satisfying J^s,\i{SiM-A') = t. 

(3) If 7^ and Si is the leftmost set of environment variables appearing in Sov, then, 
for all A G dom{l{Si)), Vs-\i[Si)i^) is well-defined. 

(4) If i, j £ [l,k], i < j, Si C Ve, Sj C Ve, r{Si)ril{Sj) C Vd {Sj is the first set of environ- 
ment variables appearing to the right of Si in Sov), {A, A') G dom{l{Si)) x dom{Si), 

\ i(Si){^) is well-defined, and Vs^ \ i(^Si){^-^') Opj then, for all A" extending A. A' 
over l{Sj), Vs^\i(Sj)iA") is well-defined. 

(5) If i,j G [l,k], i < j, Si C Vd, Sj C Vd, r{Si) n l{Sj) C Ve {Sj is the first set of 
decision variables appearing to the right of Si in Sov), {A, A') G dom{l{Si)) x dom{Si), 
^Si\i{Si)i^) is well-defined, and ^5. | ;(5.)(A.yl') = t, then, for all A" extending A. A' 
over l{Sj), Ts^\i(_s,){^") is well-defined. 

(6) For all i G [1, k] such that 5^ C Ve, Vs, \ i(s,) = 'Ps, \ i{Si)nVE \\Vd ■ 

(7) For all i G [1, k] such that 5^ C Vd, J's, \ i{S,) = ^s, \ m)nVD \\Ve- 

Proof. Wc denote by po the element in Ep such that the completion of Vv^ || Vd equals Vv^ \ \Vo®Po- 
Note that po 7^ Op, since it must satisfy ®Pi^[i,\dom{VD)\]P'^ ^ ^P' 

Lemma 1. Let {Ep,(Bp,®p) he a conditionable plausibility structure. Then, {pi ®p P2 = Op) ^ 
((pi = Op) V {p2 = Op)). 

Proof. First, if pi = Op or p2 = Op, then pi ®pP2 = Op. Conversely, assume that pi ®pP2 = Op. Then, 
if Pi >~p Op, the conditionability of the plausibility structure together with pi ®p Op = Op entails that 
P2 = Op. Similarly, if p2 >p Op, thenpi = Op. Therefore {pi®pP2 = Op) {{pi = 0p)V(p2 = Op)). □ 

Lemma 2. Assume that the plausibility structure is conditionable. Let Si, S2 be disjoint subsets of 
Ve . Then, Vs^ \So_\\Vd = ^Si | S2,Vd ■ 

Proof. Note that Vsi,S2\Vd = 'T^Si\S2.yn ®p 'Ps2\Vd- Moreover, we can also write PsuS2\Vd = 
'Psi,S2 II Vb = ^Si I S2 II ®p \\Vo = 'Psi\S2\\Vo <^p | Vo ■ ^^t A be an assignment of V. 
If 7^5^ 52 1 Vd (^) ^p ^S2|V'd(^)7 then the conditionability of the plausibility structure entails that 
^Si|S2,Vd(^) = 'Psi\S2\\Vni^)- Otherwise, Psi,S2\Voi'^) = ^S2|Vd(^)i which also entails that 
^Si,S2||Vd(^) = ^S2||yD(^)- In this case, Vs,\S2,VniA) = ^Si | S2 1 1 (^) = Ip- Therefore, 
'Psi I S2.VD = T^Si \S2\\Vo- n 

(1) Assume that Si Q Ve and Vsi\i{Si){A) is well-defined. Then, Vsi\i{Si){A) is a plausibility 
distribution over 5*^. Hence, ®PAi^iiom(s ) '^Si I '(•S'i)(^-^') ^ ^p^ which implies that there exists 
at least one A' G dom{Si) such that Vs^ \ i(Si){A.A') ^ Op. 

(2) Proof similar to point (2). 

(3) Assume that Ve 7^ 0. Let Si be the leftmost set of environment variables appearing in Sov and 
let A G dom{l{Si)). Since l{Si) n Ve = 0, we can write Vi(^Si){A) = ffipy_,(s,) ^Vb^Vd (^) = 
®PVD-i(s,)i®PVE 'PvEyoiA)) = ®PVo~i(s.)P^ Op. Therefore, Vs,\i{s^){A) is weU-defined. 
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(6) Let IsiSi) = l{Si) n Ve and loiSt) = l{Si) n Vd- For a set of variables 5", we denote by dciS) 
the set of variables in V that are descendant in the DAG G of at least one variable in S. 

First, rs,,i^is,)\\Vn = ©PyE-(S.u/E(S,))^^«IIVo = ®pVe-{S,uIe{s.)) (®pp,6P -Pj)- By defini- 
tion of a query, variables in VEndoiVD —loiSi)) do not belong to SiUlE{Si) (the environment 
variables that are descendants of as-yet-unassigned decision variables are not assigned yet). 

Thus, Vs,,Ie{S,)\\Vd = ®PVE-{S,yME{S^)UdG(VD~lD(S,)))^®PP,eFact{c),c<^VEndG{VD-lD(Si))-^j)- 

The last equality is obtained by successively eliminating the environment components included 
in dQ{Vj:,~lD{Si)) (using the normalization conditions). As the scope of a plausibility function 
Pj S Fact(c) is included in cUpaG(c), this equality entails that Vsi,iEiSi) \\ Vd does not depend 
on the assignment of Vd - loiSi). Morever, 'Pi^(s^) \\Vo = ©s, 'Psi,iE{s,) \\Vd does not depend 
on the assignment of Vd - lD{Si) too. As 'Ps^\iE{s,)\\Vn = max{p G Ep\PsidE(Si)\\VD ^ 
p ®p 'PiEiSi) II VdIj this also entails that Vs^ \ lEiSi) \\ Vn does not depend on the assignment of 
VD-lD{Si). It can be denoted 'Psi\iE(Si)\\iD(Si)- 
We now show that P^. | i(g.) = Vs, \ Ie(s^) \\iD(Si)- First, note that 

^S.,i(S.) = ®PVn-ln(S,) 'Ps,,Ie{S,),Vo = ®PVn-lD{S.)('^S, \lE{S,)yo ®p'^lE{S,),Vn) 

= ®PVo-iD{s,)'^^s,\iB(Si)\\Vo®p'^iE{s.)yn) (using Lemma 2) 

= 'Ps,\Ie(S.)\\Ve ®P (®PVu-lD{S,)'^lE{S.),Vn) 

(since Vs^ \ iE{Si) \ \ Vo does not depend on the assignment of Vd — lD{Si)) 

= 'Ps,\lE{Si}\\VD '^p'PliS,)- 

Let A be an assignment of V. 

— IiVsi,i(Si){^) 'Pi{Si){^)i then the conditionability of the plausibility structure directly 
entails that Ps, | z(s,)(^) = ^s. I lE(Si) \\Vd (^)- 

- Otherwise, 7's,,i(s.)(^) = 'Pi{Si){A)- In this case, T's, | i(Si) (^) = Ip- Next, as - 
l{Si) = {VD-lD{Si))li{VE ~lE{Si)), observe that 'Pi(Si) = ®Pv-i(Si)^^VE \\Vo '^pPo) = 
®PVn-iD{S.)^^iE(s,)\\Vn®pP^)- Similarly, wc have Pg^jfs^) = ®pv-[s,iii(s,)){'PvE\\Vn®P 
Po) = ®PVo~iD(SO^^SidE(Si)\\VD ®pPo)- As 7's,,i(s,)(^) = 'PiiSi){A-), we can infer that 

®pPo) — ®pvE-iDiSi)^'^Si,iE{Si)\\VDi^) ®pPo)- As neither 
PiE(Si) II Vn nor Vsi^EiSi) \\ Vd depends on the assignment of Vd — iDiSi), this entails that 
'PiE{S,)\\Vni^) ®P i®PVo-io(s,)P°^ = 'Ps„iEiS,)\\Voi^) ®P i®PVn-io{S,) Po'l ' humming 
this equation \dom{lD{Si))\ times gives Psi,iE{Si) HVDi"^) = 'PiE(Si) WVoi^)' and thus 
'Ps,\iE{s.)\\VoiA) = lp = Ps.[iisM^)- 

(7) Proof similar to point (6). 

(4) Let i,j e [1, k] such that i < j, Si C Ve, Sj C Ve, and r{Si)r]l{Sj) C Vd (Sj is the first set of 
environment variables appearing to the right of Si in Sov). Let {A, A') G dom{l{Si)) x dom{Si) 
such that 'Ps,\i{s,){A) is well-defined (i.e. Pj(s.)(A) ^ Op) and Vs^\i(s,){A.A') ^ Op. Let A" 
be an extension of A. A' over l{Sj). We must show that | ;(5^.) (A") is well-defined, i.e. 
that 'Pi(Sj){.A") ^ Op. As 'Ps,\i(s,){A.A') ^ % and 'Pi(Si){A) ^ Op, Lemma 1 implies that 
Vsi,i(Si){A-A') 7^ Op. Similarly to the proof of point (6), it is possible to show that Vi[Sj) does 
not depend on the assignment of l{Sj) — {Si U l{Si)). Therefore, for every A" extending A. A' 
over l{Sj), ®pii^Sj)-(s,vji{s,))'^i(Sj)iA") + %, which implies that P,(5^.)(A") ^ Op. 

(5) Proof similar to point (4), except that plausibilities are replaced by feasibilities and decision 
variables are replaced by environment variables. 

□ 
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Theorem 2 If the plausibility structure is conditionable, then, for all queries Q on a PFU 
network, Sem-Ans{Q) = Op-Ans{Q) and the optimal policies for the decisions are the same 
with Sem-Ans{Q) and Op-Ans{Q). 

Proof. Let Afr be an assignment of the set of free variables Vfr such that Tv;,.{Afr) = /• The 
semantic definition gives {Sem-Ans{Q)){Afr) = 0- Given that J-Vjri-^fr) — Vy-V/,. ^VE-Voi-^fr) = 
Vy-yj:, \\Ve i^fr) = Vy_y^.^ (Af.gf Fi{Afr)) (since the completion of Ty^ || Ve gives \\Ve ^ 
•^Vd.Ve), we can infer that for every complete assignment A" extending Afr, /^FiGF Fi(A") = f and 
iAF-eFF,{A"))*{(^pp^^pP,{A"))®p„ (®«i/.6C/C^»(^")) = 0- As min(0,0) = max(0,0) = Offi„0 = 
0, this entails that {Op-Ans{Q))(Afr) = too. 

We now analyze the case J-Vj^{Afr) = t. We use A" to denote a complete assignment which 
must be considered with the semantic definition. Using the properties: 

• P ®pu min(ui, U2) = min(p ®p„ Ui,p®pu U2) (right monotonicity of ®p„), 

• P ®pu max(ui, U2) = max(p (g)pu ui,p (Eipu U2) (right monotonicity of (Eipu), 

• P ®pu {ui ©ti U2) = {p ®pu ui) ®u (p ®pM U2) (distributivity of (8)p„ over 

• Pi ®pu {P2 ®pu u) = {pi (g)p P2) ®pu u, 

we can "move" all the Pg. \ i{^Si){A.A') to get, starting from the semantic definition, 

\i®P^e[lM.s.^VE'^sMs.)){A")®puUv{A") 
on the right of the elimination operators. 

We now prove that this quantity equals Vve \ Vn {A") ®pu Uv{A"). Let S be the rightmost set 
of quantified environment variables. The chain rule enables us to write Vve \ Vn ~ | iEiS),Vo ®p 
'PiE{s)\Vn^ where Ie{S) = 1{S) D Ve- Moreover, using Lemma 2 and Proposition 8(6), we can 
write Vs \ Ie{S),Vd = T^S \ Ie{S) \\Vd = | i(S) ■ Therefore, Vve \Vd = | i{S) ®p 'Pie{S) \ Vd ■ Recur- 
sively applying this mechanism leads to: Vve \ Vn = k] s cVe ^■5* I '(S;)- Therefore, we obtain 
'Pve I Vn {A") (Xipu l^v{A") on the right of the elimination operators. 

The semantic definition of the query meaning can be simplified a bit, thanks to Lemma 1. This 
lemma implies that conditions like Vs\i(s){^-A') ^ Op, which are used only when Vi(s){A) ^ Op, are 
equivalent to rs,i(s){A.A') 7^ Op, since Vs,i{s){A.A') = Vs\i{s){A.A') ®p Vi(s)iA). As a resuh, the 
operators ®uA'€dom{S),Vsins)iA.A')^Op can be replaced by ®uA'edoTn{s),Vs,ii,s){A.A')=^Op- Similarly, in 
the eliminations '[ninA'^dom{S),J^s\i(s)iA.A')=t, the conditions J-s \i{s)iA.A') = t can be replaced by 
^s,i{S){A-A') = t. The same holds for the eliminations with Tt\a.yiaedom(x,),j^s\i(s)(A.A')=t- 

We now start from the operational definition and show that it can be reformulated as above. The 
operational definition applies a sequence of variable eliminations on the global function (A^^gF ^i) * 
{®PP-^P Pi) ®pu [f^u^f^u Ui), which also equals Tv^, \ Ve * ^Ve | Vd ®pu ^v- Let S be the leftmost 
set of quantified decision variables. Let A be an assignment of 1{S). Assume that S is quan- 
tified by min. Let Aq S dom{S) such that J-s^i{s){A.Ai^) ~ f. It can be inferred that for 
all complete assignment A" extending A.Aq, !Fve,Vd{A") — f, and consequently ^y^|yg,(A") = 
/. This implies that Jfy^|y^(A") ★7'y^|y^(A"j ®p„ Wy(A") = 0. Given that min(0,0) = 
max(0, 0) = ©ti = 0, we obtain Qo{M, Sov, A.Aq) = 0- As min{d, 0) = d, this entails 
that mmA'edo,n{S)QoiJ^,Sov,A.A') = mmA'edorn(s)-{Ao}Qo{J^,Sov,AA'). Thus, inmA'edom{S} 
can be replaced by '[ni'n-A'£dom(S),:Fs i(s)iA.A')=t (as J-'vf^(A) = t, there exists at least one assignment 
A' € dom{S) such that !Fs,i(s){^-A') = t). The same result holds if S is quantified by max. Ap- 
plying this mechanism to each set of quantified decision variables from the left to the right of Sov, 
we obtain that minA' edomiS) and max^' edoni{s} can be replaced by miiiA' (zdo7n{s),3^s.i{s){A.A'}=t and 
^&'y^A'edom{S),J^s i{s){A.A')=t respectively. Moreover, it can be shown that for every complete assign- 
ment A" considered in the corresponding transformed operational definition, J-yo 1 Ve(^") — t- It is 
thus possible to replace 1 Ve (^") * 1 Vd (^") ^pu Uv{A") by Vve \ Vn (^") ®pu Uv{A"). 
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We now transform each ®uA'<^dom(S) Qo{N , Sov, A. A') so that it looks hke the exprcssfon in the 
semantic definition. Let 5* be the leftmost set of quantified environment variables. Let A be an assign- 
ment of 1{S). Let Aq £ dom{S) be such that Vs,i{s){^-^o) ~ Op. Then, for all complete assignments 
A" extending A.Aq, Vve \ Vd (^") = %^ and thus Vy^ j (A") ®p„ Uy{A!') = 0„. As min(0„, 0„) = 
max(0„,Oti) = 0„ ®„ 0„ — 0„, we obtain Qo{M ,Sov,A.Ai^) = 0„. As d ©„ 0„ = d, comput- 
ing ©t(^/g(;o„(5-) (5o(A/', S'oTj, j4.^') is equivalent to computing ®uA'edo)n(s)-{Ao} 

Qo(7V, Sov.A.A). 

Thus, ®«A'edom(s) can be replaced by ©«yi'edom(s),-Ps,,(s)(A.A')#Op (as ^i(s)(^) 7^ Op, there exists 
at least one assignment A G dom{S) satisfying 'Ps^i(^s){^--^') 7^ Op)- Applying this mechanism, 
considering each set of quantified environment variables from the left to the right of 5ow, we get 
®uA' £dom(s),Vs,ns){A,A')=/^Op instead of ©ti^/gdo„,(5). 

Consequently, we have found a function $ such that Sem-Ans{Q) ~ $ and Op-Ans{Q) = $. 
Moreover, the optimal policies for the decisions for Sem-Ans{Q) are optimal policies for decisions 
for Indeed, the transformation rules used preserve the set of optimal policies. The same holds 
for Op-Ans{Q) and <&. It entails that Sem-Ans{Q) = Op-Ans{Q), and that the optimal policies for 
Sem-Ans{Q) are the same as those for Op-Ans{Q). □ 

Theorem 3 Queries and bounded queries can be used to express and solve the following 
nonexhaustive list of problems: 

1. SAT framework: SAT, MAJSAT, E-MAJSAT, quantified boolean formula, stochastic 
SAT (SSAT) and extended-SSAT (Liftman et al., 2001). 

2. CSP (or CN) framework: 

• Check consistency for a CSP (Mackworth, 1977); find a solution to a CSP; count 
the number of solutions of a CSP. 

• Find a solution of a valued CSP (Bistarelli et aL, 1999). 

• Solve a quantified CSP (Bordeaux & Monfroy, 2002). 

• Find a conditional decision or an unconditional decision for a mixed CSP or a 
probabilistic mixed CSP (Fargier et al., 1996). 

• Find an optimal policy for a stochastic CSP or a policy with a value greater than 
a threshold; solve a stochastic COP (Constraint Optimization Problem) (Walsh, 
2002). 

3. Integer Linear Programming (Schrijver, 1998) with finite domain variables. 

4. Search for a solution plan with a length < k in a classical planning problem (STRIPS 
planning, Fikes & Nilsson, 1971; Ghallab et al., 2004). 

5. Answer classical queries on Bayesian networks (Pearl, 1988), Markov random fields 
(Chellappa & Jain, 1993), and chain graphs (Frydenberg, 1990), with plausibilities 
expressed as probabilities, possibilities, or At-rankings: 

• Compute plausibility distributions. 

• MAP (Maximum A Posteriori hypothesis) and MPE (Most Probable Explana- 
tion). 
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• Compute the plausibility of an evidence. 

• CPE task for hybrid networks (Dechter & Larkin, 2001) (CPE means CNF Prob- 
ability Evaluation, a CNF being a formula in Conjunctive Normal Form). 

6. Solve an influence diagram (Howard & Matheson, 1984). 

7. With a finite horizon, solve a probabilistic MDP, a possibilistic MDP, a MDP based on 
K-rankings, completely or partially observable (POMDP), factored or not (Puterman, 
1994; Monahan, 1982; Sabbadin, 1999; Boutilier et al., 1999, 2000). 

Proof. 

Lemma 3. Let {Ep, EuT®m®pu) be an expected utility structure such that is totally ordered by 
:<u. Let 7Si,S2 be a local function on Eu whose scope is Si^J 82- Then 

,0 ^ ®" 7Si,S2(</'(-4).A) = ©„niax7Si,S2. 

4>:dom{S2)^dom(Si) A<^dom[S2) S2 ^"1 

Moreover, ip : dom{S2) —> dom{Si) satisfies (maxsj 75^^32 )(^) = 7S'i,S2 (^(^)-^) /"'^ ^ S 

dom{S2) maX0.dom(S2)-dom(Si) ffi«yledom(S2) 7Si,S2(</'(^)-^) = ®uAedo7n{S2)lSi,S2i'ip{A).A). In 

other words, the two sides of the equality have the same set of optimal policies for Si . 
Proof. Let 00 : dom{S2) — > dom{Si) be a function such that 

^^^4,:dom(S2)^dom(Si) ®uA£dom{S2) 7Si .Sa ('/'(^) ■^) = ®uA^dom{S2) 7Si ,^2 ('/'O (^) ■^) ■ 

Given that, for all A € dom{S2), jSi,S2{4'oiA).A) ^„ ^3.'KA'£dom{Si) 7Si,S2(^'-^)i the monotonicity 
of ©„ entails that ®uAedoni{S2) 7Si,S2 (0o(^)-^) ffi«Aedom(S2) max^'Gdom(Si) 7Si,S2 (^'•^)- Thus, 

maX0.dom(S2)^dom(Si) ®uAGdom{S2) 7Si ,^2 ('/'(^) •^) ^« ®uS2 maxgi 7Si,S2- 

On the other hand, let 'i/'o : dom{S2) — *■ dom{Si) be a function such that \/A G dom{S2), 
(maxsi 7Si,s2)(^) 7Si,S2 ('0o(^)-^)- Then, 

©„52 maxsj 7Sj,52 = 7Si,S2(V'o(^)-^) ^« max ®u jSi,S2{<l^{A).A). 

AGdom(S2) ct,:dom{S2)^dom(Si) AGdom{S2) 

The antisymmetry of :<u implies the required equality. The equality of the set of optimal policies 
over Si is directly implied by the equality. □ 



We now give the proof of the theorem, which uses for some cases the previous lemma. 

1. {CSP based problems, Mackworth, 1977) 

Let us consider a CSP over a set of variables V and with a set of constraints {Ci, . . . , Cm}. 

(a) ( Consistency and solution finding) Consistency can be checked by using the query Q = 
{Af, (max, V)), where = (F, G, 0, 0, U) (all variables in V are decision variables. G is 
reduced to a unique decision component containing all variables, and U = {Ci, . . . , Cm}), 
and where the expected utility structure is boolean optimistic expected conjunctive 
utility (row 6 in Table 1). Computing Ans{Q) = maxy (Ci A ... A Cm) is equivalent to 
checking consistency, because Ans{Q) = t iff there exists an assignment of V satisfying 
Ci A. . .AC„i, i.e. iff the CSP is consistent. In order to get a solution when ^ns((5) =t, it 
sufhces to record an optimal decision rule for V. Integer Linear Programming (Schrijver, 
1998) with finite domain variables can be formulated as a CSP. 

(b) {Counting the number of solutions) The expected utility structure considered for this 
task is probabilistic expected satisfaction (row 2 in Table 1). The PFU network is 
A/" = (y, G,P,%,U), where all variables in V are environment variables, G is a DAG with 
a unique component cq , P ^ {l/^po}, (po being a constant factor equal to |(ioTO(y)| 
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such that Fact{cQ) ~ {^o}, smd U = {Ci, . . . , Cm}- Imphcitly, l/(po specifies that the 
complete assignments are equiprobable. It enables the normalization condition "for all 
c e Ce{G), 

®Pc®pPi^Fact(c)Pi — Ip" to be satisfied, since X]y (l/Mom(F)|) — 1. The 
query to consider is then Q ~ {M , (+, V)). It is not hard to check that this satisfies the 
conditions imposed on queries and Ans{Q) = (l/'/'o x (C'l x . . . x Cm)) gives the 
percentage of solutions of the CSP. (^o x Ans{Q) gives the number of solutions. 

2. [Solving a Valued CSP (VCSP), Bistarelli et al, 1999) 

In order to model this problem, the only difficulty lies in the definition of an expected utility 
structure. In a VCSP, a triple [E, (g, '^) called a valuation structure is introduced. It satisfies 
properties such as (iJ, ®) is a commutative semigroup, ;^ is a total order on and E has 
a minimum element denoted T. The expected utility structure to consider is the following 
one: (£^p, ®p, Op) = ({i, /},V,A), (-E'u,®^) = (-E, ®), and the expected utility structure is 
[Ep, Eu,(Bu, ®pu), with ©„ = min and defined by ''false ®pu u = T and true ®pu u = 
u" (it is not hard to check that this structure is an expected utility structure). Next, the 
PFU network is TV = (V, G, 0, 0, C/), where V is the set of variables of the VCSP, G is a 
DAG with only one decision component containing all the variables, and U contains the soft 
constraints. The query Q = (min, V) enables us to find the minimum violation degree of the 
soft constraints. A solution for the VCSP is an optimal (argmin) decision rule for V. 

3. (Problems from the SAT framework, Littman et al., 2001) 

In the SAT framework, queries on a conjunctive normal form boolean formula (j) over a set 
of variables V = {xi, . . . , a;„} are asked. Let us first prove that an extended SSAT for- 
mula can be evaluated with a PFU query. An extended SSAT formula is defined by a 
triple ((/), 9, q) where </> is a boolean formula in conjunctive normal form, is a threshold 
in [0, 1], and q — [qixi) . . . [qnXn) is a sequence of quantifier/variable pairs (the quantifiers 
are 3, V, or 9\; the meaning of 9\ appears below). If we take f ^ t, the value of un- 
der the quantification sequence q, val{4>,q), is defined recursively by: (i) val{(j),$) = 1 if 
is t, otherwise; (ii) val{(j), {3x) q') = maxx val{4>,q')'-, (iii) val{4>,{\/x) q') ~ minx val{4>,q'); 
(iv) val{(j), {9\x) q') = 0.5 • wa^(0, g'). Intuitively, the last case means that 9\ quantifies 
boolean variables taking equiprobable values. An extended SSAT formula (0, 0, q) is t iff 
val{(j),q) > 9. If S denotes the set of variables quantified by 51, an equivalent definition of 
val{(j),q) is: (i') val{(j),%) = O.Sl"^' if is t, otherwise; (n') val{4>,{3x) q') = max^: uaZ(0, g'); 
(iii') val{4),iyx)q') = miiix val{(j),q'); (iv') val{4>T i^x) q') = 'Y^^val{4>,q'). This second defi- 
nition proves that val{4>,q) can be computed with the PFU query defined by: (a) expected 
utility structure: probabilistic expected satisfaction (row 2 in Table 1); (b) PFU network: 
M = {V, C, P, 0, U), with V the set of variables of the formula (the decision variables are 
the variables quantified by 3 or V), G a DAG without arcs, with one decision component per 
decision variable and a unique environment component containing all variables quantified by 
9\, P = {(fo}, (po being a constant factor equal to 0.5'^^', and U the set of clauses of 0; (c) 
query: Q ~ {J\f, Sov), Sov being obtained from q by replacing 3, V, and 9\ by max, min, and 
-|- respectively. Then, Ans{Q) = val{(j), q), which implies that the value of an extended SSAT 
formula (0, 9, q) is the value of the bounded query (A/", Sov, 9). 

SSAT is a particular case of extended-SSAT and is therefore covered. SAT, MAJSAT, E- 
MAJSAT, QBE are also particular cases of extended SSAT. As a result, they are instances 
of PFU bounded queries. More precisely, SAT corresponds to a bounded query of the form 
Q = (TV, (max, T^), 1); MAJSAT ("given a boolean formula over a set of variables V, is it 
satisfied for at least half of the assignments of F" ) corresponds to a bounded query of the 
form [M, (-|-,y),0.5); E-MAJSAT ("given a boolean formula over V = Ve^Vd, does there 
exist an assignment of Vd such that the formula is satisfied for at least half of the assignments 
of Ve?") corresponds to a bounded query of the form (TV, (max, Vd).(+, Ve), 0.5); QBF 
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corresponds to a bounded query in which max over existcntiaUy quantified variables and min 
over universally quantified variables alternate. 

4. {Solving a Quantified CSP (QCSP), Bordeaux & Monfroy, 2002) 

A QCSP represents a formula of the form QiXi . . . QnXn (Ci A ... A Cm), where each Qi is a 
quantifier (V or 3) and each Ci is a constraint. The value of a QCSP is defined recursively as 
follows: the value of a QCSP without variables (i.e. containing only t, /, and connectives) is 
given by the definition of the connectives. A QCSP 3x qcsp is t iff either qcsp{{x,t)) = t or 
qcsp({x, /)) = t. Assuming / ^ i, it gives that 3x qcsp is t iff max^. qcsp = t. A QCSP Vx qcsp 
is t iff qcsp{{x,t)) = t and qcsp((x, f)) = t. Equivalently, Vx qcsp is t iff mhix qcsp = t. It 
implies that the value of a QCSP is actually given by the formula op{Qi)xi ■ ■ ■ op{Qn)x„ {Ci A 
... A Cm), with op{3) ~ max and op{\f) = min. It corresponds to the answer to the query 

(A/", {op{Qi),xi) {op{Qn), Xn)), whcrc N = {V, G, 0, 0, U) {V is the set of variables of the 

QCSP, G is a DAG with only one decision component containing all variables, and U is the 
set of constraints), and where the expected utility structure is boolean optimistic expected 
conjunctive utility (row 6 in Table 1). 

5. [Solving a mixed CSP or a probabilistic mixed CSP, Fargier et al., 1996) 

A probabilistic mixed CSP is defined by (i) a set of variables partitioned into a set W of 
contingent variables and a set X of decision variables; an assignment Aw of W is called 
a world and an assignment Ax of X is called a decision; (ii) a set C = {Ci, . . . ,Cm} of 
constraints involving at least one decision variable; (iii) a probability distribution Pw over 
the worlds; a possible world Aw (i.e. such that Pw{Aw) > 0) is covered by a decision Ax iff 
the assignment Aw -Ax satisfies all the constraints in C. 

On one hand, if a decision must be made without knowing the world, the task is to find 
an optimal non- conditional decision, i.e. to find an assignment Ax of the decision variables 
that maximizes the probability that the world is covered by Ax ■ This probability is equal to 
Eaw I {Cix...xc^){Ax,A„)=i Pw{Aw) = 'Ew i^w X Ci X ... X Cm)- As a result, an optimal 
non-conditional decision can be found by recording an optimal decision rule for X for the 
formula maxx i^w x Ci x . . . x Cm)- The previous formula actually specifics how to 
solve such a problem with PFUs. The algebraic structure is probabilistic expected additive 
utility (row 2 in Table 1), the PFU network is A/" = {V, G, P, 0, U), with Vd = X,Ve = W, 
G a DAG without arc, with one decision component X and a set of environment components 
that depends on how Pw is specified, P is the set of multiplicative factors that define Pw, 
and finally U = {Ci, Cm}- The query is then Q = [M, (max, X).(+, W)). 

On the other hand, if the world is known when the decision is made, the task is to look for an 
optimal conditional decision, i.e. to look for a decision rule (pQ : dom[W) — > dom{X) which 
maximizes the probability that the world is covered. In other words, the goal is to compute 

^^^4,:dom{W)^dom{X) J2Aw&dom{W) | (Ci X . . . X C„)( Aw ■</>( Aw )) = 1 ^VV (Aw) = 

T^a^^:dom{w)-^doniix) J2 A.y edoni{w} (P^ X Ci X . . . X Cm) [Aw .(j}{Aw)) - Due to Lemma 3, it 
also equals J2w m^^x (Pw x Ci x . . . x Cm), and 0o can be found by recording an optimal 
decision rule for X. It proves that the query Q = {Af, (+, VF).(max, X)) enables us to compute 
an optimal conditional decision. 

With Mixed CSPs, Pw is replaced by a set K of constraints defining the possible worlds. The 
goal is then to look for a decision, either conditional or non-conditional, that maximizes the 
number of covered worlds. This task is equivalent, ignoring a normalizing constant, to find a 
decision that maximizes the percentage of covered worlds. This can be solved using the set 
of plausibility functions P = K U {Nq}, with Nq a normalizing constant ensuring that the 
normalization condition on plausibilities holds. Nq is the number of possible worlds, but it 
does actually not need to be computed, since it is a constant factor and we are only interested 
in optimal decisions. 
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6. {Stochastic CSP (SCSP) and stochastic COP (SCOP), Walsh, 2002) 

Formally, a SCSP is a tuple {V,S,P,C,9), where F is a list of variables (each variable x 
having a finite domain dom{x)), S is the set of stochastic variables in P = {Ps | s G 5} 
is a set of probability distributions (in a more advanced version of SCSPs, probabilities over 
S may be defined by a Bayesian network; the subsumption result is still valid for this case), 
C = {Ci, . . . , Cm} is a set of constraints, and 6* is a threshold in [0, 1]. 

A SCSP-policy is a tree with internal nodes labeled with variables. The root is labeled with 
the first variable in V , and the parents of the leaves are labeled with the last variable in 
V . Nodes labeled with a decision variable have only one child, whereas nodes labeled with 
a stochastic variable s have \dom{s)\ children. Leaf nodes are labeled with 1 if the complete 
assignment they define satisfies all the constraints in C, and with otherwise. With each leaf 
node can be associated a probability JIsgs ^s(^s)' where As stands for the assignment of S 
implicitly defined by the path from the root to the leaf. The satisfaction of a SCSP-policy 
is the sum of the values of the leaves weighted by their probabilities. A SCSP is satisfiable 
iff there exists a SCSP-policy with a satisfaction of at least 9. The optimal satisfaction of a 
SCSP is the maximum satisfaction over all SCSP-policies. 

For the subsumption proof, we first consider the problem of looking for the optimal sat- 
isfaction of a SCSP. In a SCSP-policy, each decision variable x can take one value per 
assignment of the set predg^x) of stochastic variables which precede x in the list of vari- 
ables V. Instead of being described as a tree, a SCSP-policy can be viewed as a set of 
functions A = {0^ : dom(preds{x)) —> dom{x)),x ^ V — S}, and its value is val{A) = 
EAsGdom(S) (rises -P" X nc,ecC'0(^s-(^^^_^(/'''(As))). The goal is to maximize the previ- 
ous quantity among the sets A. Let y be the last decision variable in V, and let be the set 
of local functions (/>^ : dom{preds{y)) — > dom{y) defining a decision rule for y. Then 

max.a;(A)= max ( E E Ps x U C^)(^s ■i^^^_^MAs))). 

Asedom{preds{y)) S-preds{y} seS CiEC 

By Lemma 3, the previous quantity also equals: 

Epred^y)) J2s-pred,{v) {UseS X Uc.&c ^0 ' ^ recursivc apphcation of this mecha- 
nism shows that the answer Ans{Q) to the query Q described below is equal to the optimal 
satisfaction of a SCSP: 

• expected utility structure: row 2 in Table 1 (probabilistic expected satisfaction); 

• PFU network: Af ^ {V' , G, P, 0, U), with V the set of variables of the SCSP; Ve ^ S 
and Vd ^ V — S; G is a DAG without arcs, with one component per variable; P = 
{Ps I s e S}; Fact{{s}) = {PJ; U is the set of constraints of the SCSP; 

• query: Q =(7V, Sov), with Sov=t{V) {V is the list of variables of the SCSP), t{V) being 

recursively defined by t(0) = and t{x.V") = ( ^"^l^'jS^lln ^ ■ ■ 
•' ^ ' I (max, {a;}).t(\/ ) otherwise 

An optimal SCSP-policy can be recorded during the evaluation of Ans{Q). The satisfiability 
of a SCSP can be answered with the bounded query (M,Sov,6). Again, a corresponding 
SCSP-policy can be obtained by recording optimal decision rules. 

With Stochastic Constraint Optimization Problem (SCOP), the constraints in C are additive 
soft constraints. The subsumption proof is similar. 

7. [Classical planning problems (STRIPS planning), Fikes & Nilsson, 1971; Challab et ai, 2004) 
In order to search for a plan of length lesser than fc, we can simply model a classical plan- 
ning problem as a CSP. Such a transformation is already available in the literature (Challab 
et al., 2004). However, we can also model a classical planning problem more directly in the 
PFU framework. More precisely, the state at one step can be described by a set of boolean 
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environment variables. For each step, there is a unique decision variable whose set of val- 
ues corresponds to the actions available. Plausibility functions are deterministic and link 
variables in step t to variables in step t + 1 (these functions simply specify the positive and 
negative effects of actions). The initial state is also represented by a plausibility function 
which links variables in the first step. Feasibility functions define preconditions for an action 
to be feasible. They link variables in a step t to the decision variable of that step. Utility 
functions are boolean functions which describe the goal states. They hold over variables in 
step k. In order to search for a plan of length lesser than fc, the sequence of elimination is a 
max-elimination on all variables. The expected utility structure used is the boolean optimistic 
expected disjunctive utility. 

8. [Influence diagrams, Howard & Matheson, 1984) 

We start from the definition of influence diagrams of Section 3. With each decision variable 
d, we can associate a decision rule 6"^ : dom{paG{d)) — > dom{d). An influence diagram policy 
(ID-policy) is a set A ~ {S'^ \ d £ D} of decision rules (one for each decision variable). The 
value val{A) of an ID-policy A is given by the probabilistic expectation of the utility: 

val{A)= ((n^^lf-o(.))x(E 

Asedom{S) seS UiGU 

To solve an influence diagram, we must compute the maximum value of the previous quan- 
tity and find an associated optimal ID-policy. Using Lemma 3 and the DAG structure, it is 
possible to show, using the same ideas as in the SCSP subsumption proof, that the optimal 
expected utility is given by the answer to the query Q below (associated optimal decision rules 
can be recorded during the evaluation of Ans(Q)): 

• expected utility structure: row 1 in Table 1 (probabilistic expected additive utility); 

• PFU network: J\f = (V, G', P, 0, C/); V is the set of variables of the influence diagram, 
G' is the DAG obtained from the DAG of the influence diagram by removing utility 
nodes and arcs into decision nodes; in G', there is one component per variable; P = 
{Ps\paGis)j s G Ve} and Fact{{s}) — {Ps\paG{s)}j U is the set of utility functions 
associated with utility nodes. 

• PFU query: Q = {Af, Sov), with Sov obtained from the DAG of the influence diagram as 
follows. Initially, Sov = 0. In the DAG of an influence diagram, the decisions are totally 
ordered. Let d be the first decision variable in the DAG G of the influence diagram (i.e. 
the decision variable with no parent decision variable). Then, repeatedly update Sov 
by Sov <— Sov.{+,paG{d)).{ma.x, {d}) and delete d and the variables in padd) from G 
until no decision variable remains. Then, perform Sov <— Sov.{+, S), where S is the set 
of chance variables that have not been deleted from G. 

9. {Finite horizon MDPs, Puterman, 1994; Monahan, 1982; Sabbadin, 1999; Boutilier et al., 
1999, 2000) In order to prove that the encoding in the PFU framework given in Sections 5.6 
and 6.6 actually enables us to solve a T time-steps probabilistic MDP, we start by reminding 
the algorithm used to compute an optimal MDP-policy. Usually, a decision rule for dr is 
chosen by computing V*^ = max^^ Rs^^dT- ^s*t optimal reward which can be obtained 
in state st- At a time-step i £ a- decision rule for di is chosen by computing V*. — 
maxrf; {Rsi,di + X^s +1 ^si+i \si,di X Last, the optimal expected value of the reward, 
which depends on the initial state si, is V*^. 

Let us prove by induction that for alH G [1, T — 1], 

= maxdi J2s2 ■ ■ -iT^axrf, X^s.+i iiUk€[i,i] Ps,+, I s„dj X ((Efc6[i,i] Rs^,d,) + V*^J). 
This proposition holds for i = 1, since 

K*i = maxrfi {R,, + hi ,<ii X V*^ ) 

= maxrf, (Ps2 1 si.di X {Rs,,d, + V;J) (since J2s.2 \ s,,d, = !)■ 



481 



PrALET, VERFAILLIE, & SCHIEX 



Moreover, if the proposition holds at step i — 1 (with i > 1), then 

Vs\ = maxdi ■ •■maxd^-i Es. ((Ilfceli,*-!] ^'s^+i x ((Efce[i,,-i] Rs,,d,) + V,*)). 

Given that 

= maxd, ((Efce[i,,]i?..,dJ+E.,+i-P..+i|s,,d, x F,*^J 
= maxd, Es,+i A.+i X ((Efc6[i,z] Rs„dJ+V*^J 
(the last equality holds since Es +i ^si+i I si.di = l)i it can be inferred that 

(nfe6[l,,-l] -Psfc + i I Sfc.dfc) X ((Efcg[l,i-1] Rsk-dk) + 

= maxd, E,^_^^ ((Ilfeeli.z] Ps^+i\s,,dJ x ((Efe6[i,,:] Rs,,dJ + V*^J), 
which proves that the proposition holds at step i. This proves that it also holds at step T, and 
therefore V*_^ = Ans{Q). Furthermore, as each step in the proof preserves the set of optimal 
decision rules, an optimal MDP-policy can be recorded during the evaluation of Ans{Q). 

We now study the case of partially observable finite horizon MDPs (finite horizon POMDPs). 
In a POMDP, we add for each time step t > 1 a conditional probability distribution P^^ | 
of making observation ot at time step t given the state Sf . The value of St remains unob- 
served. We also assume that a probability distribution Pg^ over the initial state is available. 
The subsumption proof for this case is more difficult. We consider the approach of POMDPs 
which consists in finding an optimal policy tree. This approach is equivalent to compute, 
for each decision variable dt, a decision rule for dt depending on the observations made so 
far, i.e. a function : dom{{o2, ■ ■ ■ , Ot}) dom{dt). The set of such functions is denoted 
A set A = . . .(^'^^I is called a POMDP-policy. The value of a POMDP-policy 

is recursively defined as follows. First, the value of the reward at the last decision step, 
which depends on the assignment Agr^ of st and on the observations 02^t made from the 
beginning, is y(A)5^_02,...,ot (^st ■02^t) = i?ST,dT(^ST) <^'*^(C'2^t)). At a time step i, the 
obtained reward depends on the actual state As^ and on the observations 02->i made so far. 
Its expression is: 

V{/^)s,.o^,....oAAs,.02^^) 

= {Rs.,d. + Es.^i ^., + 1 I s^,d. X Eo,+i Po.^l I ! X y(A).,^,,o„...,o,^ J(A) 

where A = As-.(l)'^'{02^i)-02^i This equation and the recursive formula used to define the 
value of a policy tree for a POMDP (Kaelbling, Liftman, & Cassandra, 1998) are equiva- 
lent. Finally, the expected reward of the POMDP-policy A is V{A) = Esi x ^(A)^^. 
Solving a finite horizon POMDP consists in computing the optimal expected reward among 
all POMDP-policies (i.e. in computing V* = max^d^ ^dj-V{{(j)'^'^,...,(f>'^'^})), as well as 
associated optimal decision rules. 

Using a proof by induction as in the observable MDP case, it is first possible to prove that 
for a problem with T steps, 

V* = max^di ... Eo2,...,OT T>s^....^sT f^v 

with (3v = {Ps, X R,e[UT[Ps,+i\s,,d, X R,e[i,T[Po,+n.,^J x (E»e[i,T] Ps,,d,)- 
From this, a recursive use of Lemma 3 enables us to infer that 

V* = maxdi Eo2 rnaxd^ E03 max^g . . . Eo^ max^^ J2s,^...,st 
It proves that the query defined below enables us to compute V* as well as an optimal policy: 

• algebraic structure: probabilistic expected additive utility (row 1 in Table 1); 

• PFU network: TV = [V, G, P, ^,U); V equals {s, | i S [1, T]}U{oi\i G [2, T]} U{d,\i e 
[l,r]}, with Vd = {di \ i G [1,T]}; G is a DAG with one variable per component; a 
decision component does not have any parents, an environment component {oi} has 
{si} as parent, and a component {si+i} has {si} and {di} as parents; P = {^si} U 
{Ps,^,\s,,d,,i e [l,T ~l]}U{Po,lsA^ e [2,r]}; Fact{{si}) = {P,J, Fact{{s,+i}) = 
{Ps,+^\s.,dA, and Fact{{o^}) ^ {Po.hJi last, C/ = i G [l,r]}; 
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• PFU query: based on the DAG, a necessary condition for a query to be defined is that 
each decision di must appear to the left of the variables in {sk | fc G [t + 1, T]} U {ok \ k G 
[i + 1, r]}; the query considered is Q = (TV, Sov), with 

Sov = (max, di).(+, 02).(max, ^2) (+, OT).(max, dT)-{+, {si, . . . , st})- 

The proofs for finite horizon (PO)MDPs based on possibilities or on K-rankings are similar. 
As for the subsumption of factored MDPs, we can first argue that every factored MDP can 
be represented as a usual MDP, and therefore as a PFU query on a PFU network. Even if 
this is a sufficient argument, we can define a better representation of factored MDPs in the 
PFU framework: it corresponds to a representation where the variables describing states are 
directly used together with the local plausibility functions and rewards, which can be modeled 
by scoped functions (defined as decision trees, binary decision diagrams. . . ). 

{Queries on Bayesian networks, Pearl, 1988, Markov random fields, Chellappa & Jain, 1993, 
and chain graphs, Frydenberg, 1990) 

It suffices to consider chain graphs, since Bayesian networks and Markov random fields are 
particular cases of chain graphs. The subsumption proofs are provided for the general case of 
plausibility distributions defined on a totally ordered conditionable plausibility structure. 

(a) {MAP, MPE, and probability of an evidence) As MPE (Most Probable Explanation) and 
the computation of the probability of an evidence are particular cases of MAP (Maximum 
A Posteriori hypothesis), it suffices to prove that MAP is subsumed. The probabilistic 
MAP problem consists in finding, given a probability distribution Vy, a Maximum A 
Posteriori explanation to an assignment of a subset O oiV which has been observed (also 
called evidence) . More formally, let D denote the set of variables on which an explanation 
is sought and let e denote the observed assignment of O. The MAP problem consists in 
finding an assignment A* of D such that ma:iiAedom{D) Pd \o{^-^) = Pd \oi^* ■^)- As 
Pd I o = Pd,o/Po, we can write: 
ma.XAedom{D)PD\o{A.e) = {inaxAedo7n{D) PD,o{A.e)) / Po{e) 

= (max^gdom(D) Y.A'edom(v-{Duo)) Pv{A.e.A'))/Po{e) 
Thus, computing max^i J2v-{duO) ^v{g) is sufficient (the difference lies only in a nor- 
malizing constant). This result can be generalized to all totally ordered conditionable 
plausibility structures. 

Indeed, as ®p is monotonic, ma.yiA^dom{D)PD,o{A.e) = (niax^gdo^(£,) T'^, | o(A-e)) ®p 
Po(e)- If va&-KA£dom(D)PD,o{A-e) -<p Vo{e), then there exists a unique p € Ep such 
that ma.yiAedo7n{D) 'PD.o{A.e) ^ p^pVo{e). This gives us p = niax^£do„(£,) 7'D|o(^-e). 
Otherwise, if Tiia.yiAi^dom(D)PD,o{A.e) = 'Po{g), then we can infer that there exists 
A* G dom{D) such that VD,o(A*.e) = Vo{e), and therefore rD\o{A*.e) = \p. Thus, 
va&y.A^dom(D)PD\o{A-(i) = Ip too. This shows that determining max^£^o„(£)) 7'£)^o(^-e) 
gives max^grfo^(£,) Vd \o{A.e). 

Moreover, if A* G argmax{7'D,o(^'-e), A' G dom{D)}, then max{p G Bp \ Vd.o{A* .e) = 
P «)p Po{e)} hp max{p G Ep \ Vd,o{A-^) = P ®p ^o(e)} for aU A G doin{D). There- 
fore, an optimal assignment of D for maxjj Pd,o{s) is also an optimal assignment of D 
for max£) 7-"/) | o(e). As a result, the MAP problem can be reduced to the computation of 

max_D VD,o{e) — max/) ffipy_(£)ijo) Pv{e) — maxu {Vv ®p So) 

where 60 is the scoped function with scope O such that 6o{e') = Ip if e' = e. Op other- 
wise. We define a PFU query whose answer is Ans{Q) = max^ (Bpy_jy{'Pv So)' 

• the plausibility structure is {Ep, (Bp, ®p), the utility structure is {Eu,^u) = {Ep, ^p), 
and the expected utility structure is {Ep, ©„, <^pu) = {Ep, Ep, (Bp, (8)p); 

• PFU network: the difficulty in the definition of the PFU network lies in the fact that 
normalization conditions on components must be satisfied. The idea is that only the 
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components in which a variable in D U O is involved have to be modified. The PFU 
network is Af = (V, G, P, ^,U)\ V the set of variables of the chain graph; Vd = D 
and Ve = V — D\ G \s a. DAG of components obtained from the DAG G' of the chain 
graph by splitting every component c in which a variable in D U O is involved: such 
a component c is transformed into |c| components containing only one variable; all 
these |c| components become parents of the child components of c; for a component 
{xq\ included in one of these |c| components, if xo G -D, then {xq\ is a decision 
component; otherwise, {xq\ is an environment component, and we create a plausi- 
bility function P^, equal to a constant pq{xq) such that ©p^gj^ \dom(xo)\] Poi^o) = Ipj 
and such that Fact{{xQ}) ~ {poixo)}; P contains first the constants defined above, 
and second the factors expressing P^ | p^^, in the chain graph for the components 
c satisfying c n (£> U O) =0; last, U contains the factors expressing P^ | p^^, in the 
chain graph for the components c such that c n (Z? U O) 7^ 0, and a constant factor 
Pi{xq) satisfying pi{xo) ®p po{xo) = Ip for each component {xq} created in the 
splitting process described above, and hard constraints representing 8o\ with this 
PFU network, the local normalization conditions are satisfied, and the combination 
of the local functions equals Vy ®v 

• PFU query: the query is simply Q = (TV, (max, £)).(©„, V — D)). 

An optimal decision rule for D can be recorded during the computation of Ans{Q). 

(b) [Plausibility distribution computation task) Given a plausibility distribution Vv ex- 
pressed as a combination of plausibility functions as in chain graphs, the goal is to 
compute the plausibility distribution Vs over a set S C V. The basic formula Vs = 
ffipv-S proves that the query defined below actually computes Vs- This query shows 
the usefulness of free variables. 

• the plausibihty structure is (Ep, (Bp, the utility structure is (i^u, = {Ep, (g)p), 
and the expected utility structure is (Ep, ©„, ®p„) = {Ep, Ep, Op, (g)p); 

• PFU network: Af = {V, G, P, 0, U), with Ve = V - S,Vd = S, and with the DAG 
G and the sets P, U obtained similarly as for the MAP case; 

• PFU query: Q = {M, {®u,V - S)) 

[Hybrid networks, Dechter & Larkin, 2001 ) 

A hybrid network is a triple [G, P, F) , where G is a DAG on a set of variables V partitioned 
into R and D, P is a set of probability distributions expressing Pr\paG{r) for all r £ R, 
and P is a set of functions fpaQ(d) for all d G D (variables in D are deterministic, in the 
sense that their value is completely determined by the assignment of their parents). The 
most general task on hybrid networks is the task of belief assessment conditioned on a for- 
mula (j) in conjunctive normal form. It consists of computing the probability distribution 
of a variable x given a complex evidence (p (complex because it may involve several vari- 
ables). Ignoring a normalizing constant, it requires to compute, for all assignments {x,a) of 
Y.Aedo,n{v-{x})\<i,(A.(x,a))=tPv{Mx,a)). If C = {Ci,...,C™} denotes the set of clauses 
of (j), it also equals (Ey-{a;} (HrGfl Ip^gM) x illd^D fpacid)) x (HcgC a)). 
The query corresponding to this computation uses the probabilistic expected satisfaction struc- 
ture (row 2 in Table 1), and the PFU network M = {V, G, P, 0, U), with Ve =V,Vd = {x}, 
P = {Pr\paG(T) I r e P - {x}} U {fpacid) \d £ D - {x}}, and either P = C U {Pxipaoix)} if 
a;ePorP==CU {fpaoix)} it x E D. The query is Q (A/", {+, V - {x})). 

□ 
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